s11n is here to Save Our Data, man!
Project powered by:
s11nconvert: a tool for converting between data formats
s11nconvert is a command-line tool which ships with libs11n. It
can be used to convert s11n-saved data between any formats which
s11nlite can use.
This page demonstrates some uses of s11nconvert. Much of the material
covered here also applies to the s11nbrowser,
except that s11nbrowser has a graphical user interface.
Below is an overview of it's command-line options. All arguments must
start with either - or --, like -help or --help. Arguments and their
values may optionally be separated by '=', and a '=' is required if
the value starts with a '-'. Order of the arguments is insignificant,
and, unless noted otherwise, if an arg is repeated only the last one
print full help text.
f filename, file=filename:
specify input filename.Filename of - means stdin.
o filename, output=filenmae:
specify output filename. Filename of - means stdout.
s SERIALIZER, format SERIALIZER:
output format (i.e., a Serializer's name).
FORCE a given Serializer for INPUT. Required when reading
from non-file-based Serializers (e.g., db- or network-based ones).
DL dllname, dl dllname (Versions 1.0, 1.1.x, 1.2.1+)
Loads the given DLL. 'dl' causes an exit if the DLL cannot be
loaded, but 'DL' is tolerant of failed loads. May be used multiple
times. DLL names may be absolute file names or base filenames if those
files are in the lib path. Use -v to see information about what DLLs
get loaded this way. All DLLs are loaded before other arguments are
processed. Use the -v option to see which DLL files actually get
Dumps a list of all registered Serializers, including any
loaded via -dl, -DL, -s or -S.
z, bz: (Only if your s11n has zfstream support)
Compress output using gzip or bzip2. Only works for files,
not when writing to stdout nor reading from stdin (sorry about that).
cldebug: (Only in 1.0, not 1.1+)
Toggles on classloader debug output.
Verbose mode (more output), sent to stderr so it won't
interfere with serialized output.
1.2+: when used with -K, it prints all known serializer cookies.
1.2+: when used with -dl, it shows which DLL files it opens.
Print app and libs11n version info and quit.
Regarding the -s and -S options: the exact Serializers (data formats)
supported by s11nconvert depends on s11nlite - any which s11nlite
inherently knows, or can dynamically load, can be used. For our
purposes we will assume that the formats listed on the Serializers page are available. In
brief, they are: funxml, funtxt, simplexml, parens, compact, wesnoth
In s11n 1.0 (not 1.1+), it is possible that -dl and -DL will trigger
other help options. For example, '-dl mysql' will include help
options specific to the mysql_serializer:
s11nconvert -dl mysql_serializer --help | grep mysql
mysql_serializer: Sets the s11n/mysql database's name db server.
mysql_serializer: Sets the mysql db server.
mysql_serializer: Sets the mysql user's password.
mysql_serializer: Sets the mysql db user.
mysql_serializer: Toggles on verbose mode.
Let's start with some simple examples...
Convert a given input file to the given serializer format:
If you omit -o then stdout is used. If you omit -f then stdin is used.
Note that the -z and -bz options do not work when writing to stdout
or reading from stdin.
s11nconvert -f input_file -s output_serializer -o output_file
If you're using the bash shell, you can do the following to convert
a file to all known formats, compressing each output file using gzip
Note that the -K option might return several aliases for the
same serializer, however (the prominent example being
'51191011' and 'compact', which are both aliases for
for S in $(s11nconvert -K); do
s11nconvert -z -f myinfile -s $S -o myoutfile.$S.s11n
Now let's do something a bit more advanced: load a serializer from
a DLL. Actually, we rarely need to manually do this, because the
-s and -S options automatically load their Serializer if it is not
The -dl/-DL options can open arbitrary DLLs, but there is little use
in opening non-serializer DLLs this way.
s11nconvert -dl mysql_serializer [whatever other options...]
Let's pipe several s11nconverts together
(even if what they're doing in this example is nonsensical):
cat infile.s11n | s11nconvert -s parens | s11nconvert -s compact
Okay, enough playing around - let's do something interesting...
Let's read a node named MyDbNode from a database using
mysql_serializer, then save it to stdout using
the wesnoth format:
The -s and -S options do DLL lookups if needed. For
example, '-S mysql_serializer' will dynamically load the mysql_serializer
DLL, so we need not explicitely provide '-dl mysql'.
s11nconvert -S mysql_serializer -s wesnoth -f MyDbNode
And now let's save a file to the db:
s11nconvert -s mysql_serializer -o MyDbNode -f infile
Here's a preview of experimental support from ps11n,
which can save/load to/from http connections:
s11nconvert -f infile -s pio -o 'http://s11nserver/s11napp?node_name=mynode'
s11nconvert -o outfile -s funxml -S pio -f 'http://s11nserver/s11napp?node_name=mynode'
s11nconvert is very useful in the development of new Serializers,
as we can test them by simply invoking them via s11nconvert.
Using the -v and -dl options we can help track down failed DLL
loads when testing Serializers. If a Serializer works via
s11nconvert, it can be considered compatible with the rest of the
framework, as s11nconvert is capable of excercising all of the various
That's about all there is to it, but i will leave you with
an interesting note regarding s11nconvert as an example of
an s11nlite client application: about 95% of s11nconvert's
source code is simply there to handle the command-line options
to figure out what steps it should take. The actual work
is done in only three lines of code, which i will reproduce here,
shorn of error handling:
Note that s11nconvert never calls
s11nconvert only uses s11n containers,
not concrete Serializable types. It could have used
s11nlite::save() to get the same effect as above, however.
// load input node...
s11nlite::node_type * innode = use_infile
? s11nlite::load_node( ifname ) // load from file
: s11nlite::load_node( *is ); // load from stream
// save using the specified serializer...
s11nlite::serializer_base_type ser = s11nlite::create_serializer( fmt )
bool workie = use_ofile
? ser->serialize( *innode, ofname ) // save to file
: ser->serialize( *innode, *os ); // save to stream
It is also that simple for your software to use s11nlite!
The s11nconvert sources may be a helpful starting
point; you can find them in the s11n source tree under