s11nconvert: a tool for converting between data formats

s11nconvert is a command-line tool which ships with libs11n. It can be used to convert s11n-saved data between any formats which s11nlite can use.

This page demonstrates some uses of s11nconvert. Much of the material covered here also applies to the s11nbrowser, except that s11nbrowser has a graphical user interface.

Below is an overview of it's command-line options. All arguments must start with either - or --, like -help or --help. Arguments and their values may optionally be separated by '=', and a '=' is required if the value starts with a '-'. Order of the arguments is insignificant, and, unless noted otherwise, if an arg is repeated only the last one is accepted.

?, help:
        print full help text.
f filename, file=filename:
        specify input filename.Filename of - means stdin.
o filename, output=filenmae:
        specify output filename. Filename of - means stdout.
s SERIALIZER, format SERIALIZER:
        output format (i.e., a Serializer's name).
S INPUT_SERIALIZER:
        FORCE a given Serializer for INPUT. Required when reading
from non-file-based Serializers (e.g., db- or network-based ones).
DL dllname, dl dllname (Versions 1.0, 1.1.x, 1.2.1+)
        Loads the given DLL. 'dl' causes an exit if the DLL cannot be
loaded, but 'DL' is tolerant of failed loads. May be used multiple
times. DLL names may be absolute file names or base filenames if those
files are in the lib path. Use -v to see information about what DLLs
get loaded this way. All DLLs are loaded before other arguments are
processed. Use the -v option to see which DLL files actually get
loaded.
K, known-serializers:
        Dumps a list of all registered Serializers, including any
loaded via -dl, -DL, -s or -S.
z, bz: (Only if your s11n has zfstream support)
        Compress output using gzip or bzip2. Only works for files,
not when writing to stdout nor reading from stdin (sorry about that).
cldebug: (Only in 1.0, not 1.1+)
        Toggles on classloader debug output.
v, verbose:
        Verbose mode (more output), sent to stderr so it won't
interfere with serialized output.
	1.2+: when used with -K, it prints all known serializer cookies.
	1.2+: when used with -dl, it shows which DLL files it opens.
version, V:
        Print app and libs11n version info and quit.

Regarding the -s and -S options: the exact Serializers (data formats) supported by s11nconvert depends on s11nlite - any which s11nlite inherently knows, or can dynamically load, can be used. For our purposes we will assume that the formats listed on the Serializers page are available. In brief, they are: funxml, funtxt, simplexml, parens, compact, wesnoth and expat.

In s11n 1.0 (not 1.1+), it is possible that -dl and -DL will trigger other help options. For example, '-dl mysql' will include help options specific to the mysql_serializer:

s11nconvert -dl mysql_serializer --help | grep mysql
mysql-db:
        mysql_serializer: Sets the s11n/mysql database's name db server.
mysql-host:
        mysql_serializer: Sets the mysql db server.
mysql-password:
        mysql_serializer: Sets the mysql user's password.
mysql-user:
        mysql_serializer: Sets the mysql db user.
mysql-verbose:
        mysql_serializer: Toggles on verbose mode.

Let's start with some simple examples...

Convert a given input file to the given serializer format:

s11nconvert -f input_file -s output_serializer -o output_file

If you omit -o then stdout is used. If you omit -f then stdin is used. Note that the -z and -bz options do not work when writing to stdout or reading from stdin.

If you're using the bash shell, you can do the following to convert a file to all known formats, compressing each output file using gzip format:

for S in $(s11nconvert -K); do
  s11nconvert -z -f myinfile -s $S -o myoutfile.$S.s11n
done

Note that the -K option might return several aliases for the same serializer, however (the prominent example being '51191011' and 'compact', which are both aliases for 'compact_serializer').

Now let's do something a bit more advanced: load a serializer from a DLL. Actually, we rarely need to manually do this, because the -s and -S options automatically load their Serializer if it is not already loaded.

s11nconvert -dl mysql_serializer [whatever other options...]

The -dl/-DL options can open arbitrary DLLs, but there is little use in opening non-serializer DLLs this way.

Let's pipe several s11nconverts together (even if what they're doing in this example is nonsensical):

cat infile.s11n | s11nconvert -s parens | s11nconvert -s compact

Okay, enough playing around - let's do something interesting...

Let's read a node named MyDbNode from a database using mysql_serializer, then save it to stdout using the wesnoth format:

s11nconvert -S mysql_serializer -s wesnoth -f MyDbNode

The -s and -S options do DLL lookups if needed. For example, '-S mysql_serializer' will dynamically load the mysql_serializer DLL, so we need not explicitely provide '-dl mysql'.

And now let's save a file to the db:

s11nconvert -s mysql_serializer -o MyDbNode -f infile

Here's a preview of experimental support from ps11n, which can save/load to/from http connections:

// save:
s11nconvert -f infile -s pio -o 'http://s11nserver/s11napp?node_name=mynode'
// load:
s11nconvert -o outfile -s funxml -S pio -f 'http://s11nserver/s11napp?node_name=mynode'

s11nconvert is very useful in the development of new Serializers, as we can test them by simply invoking them via s11nconvert. Using the -v and -dl options we can help track down failed DLL loads when testing Serializers. If a Serializer works via s11nconvert, it can be considered compatible with the rest of the framework, as s11nconvert is capable of excercising all of the various de/serialization operations.

That's about all there is to it, but i will leave you with an interesting note regarding s11nconvert as an example of an s11nlite client application: about 95% of s11nconvert's source code is simply there to handle the command-line options to figure out what steps it should take. The actual work is done in only three lines of code, which i will reproduce here, shorn of error handling:

 // load input node...
s11nlite::node_type * innode = use_infile
                               ? s11nlite::load_node( ifname ) // load from file
                               : s11nlite::load_node( *is ); // load from stream
 // save using the specified serializer...
s11nlite::serializer_base_type ser = s11nlite::create_serializer( fmt )
bool workie = use_ofile
              ? ser->serialize( *innode, ofname ) // save to file
              : ser->serialize( *innode, *os ); // save to stream

Note that s11nconvert never calls s11nlite::de/serialize<>(), because s11nconvert only uses s11n containers, not concrete Serializable types. It could have used s11nlite::save() to get the same effect as above, however.

It is also that simple for your software to use s11nlite! The s11nconvert sources may be a helpful starting point; you can find them in the s11n source tree under src/client/s11nconvert/*.?pp.

Main menu:

Download from:

Libs/apps/utils:

SourceForge:

More Resources:

Related sites:

s11nconvert: a tool for converting between data formats