MySQL Serializer

mysql_serializer ("MyS", for short)

The s11n::io::mysql_serializer class is powered by a C++ library for mysql written by Anders Hedström. He's got an impressive set of utility libraries on his site. Using MyS does not require his mysql library: this tree comes with a slightly hacked copy of his code.

• The core library is 100% ignorant of it, and can load it by being given it's class name.
• It proves that clients can provide their own i/o handlers without touching the core library. (Previously this was pure theory.) A client-written Serializer becomes a full-rights player in the framework, just like the Serializers shipped with the library.
• It demonstrates that the previous point can be accomplished with relatively little effort. i spent about 8 hours coding it and touching it up, including the learning curve for the mysql-related code, and tweaking s11nconvert and the core lib to be able to import DLLs and arbitrary Serializers this way.
• It demonstrates that non-file-based Serializers are a realistic option, and are treated with the same API transparency as their file-based cousins.
• Demonstrates the ability to chain Serializers together to do more complex work. This one, e.g., when serializing to a STREAM it saves a record to the db and saves proxying information to the stream using a different Serializer (i.e., one that natively supports streams). That proxying info can later be loaded just like any s11n data, but it will direct MyS to load the db object from the database.

Source code...

Big Fat Caveats!

• This code is brand new (only a few hours old, as of the initial release (15 Aug 2004)), and probably contains some notable bugs. That said, i have been able to use it with s11n for a wide variety of s11n data and haven't seen any particular problems yet. Theoretically, saving very complex strings may cause some Grief, but this isn't proven yet (and should be trivial to fix when it shows up).
• This code does no error checking for things like a broken db connection. It does properly recognize when it cannot connect at all, but if a connection dies during the life of the app Grief will probably result. Since the run-time of s11nconvert is normally only a few seconds, this has not yet become a practical concern, but it easily could for longer-running s11n applications. (To be fixed, maybe. Patches are of course welcomed.)
• Using it requires a running mysql server and having a valid login for it, with write access to the database (see below). It is possible to create a user just for this purpose, and configure the build tree with --mysql-user=your_s11n_user. That does not create the user or add any rights for it - it only sets the default user name used by the Serializer when logging in to the db. There are also the following related configure options: --mysql-db, --mysql-password, and --mysql-host, all of which are supported by both the configure script and s11nconvert.
• MyS is horribly slow: 5-7 nodes per second is about it's peak, even when using a local mysql server on a single-user machine.
• It doesn't yet allow very fine control over the db connections. Patches would be most appreciated (and feel free to mutilate the db-related interface as much as you need to, just not the Serializer interface!).
• Minor bug: there appears to be some odd whitespace translation being done at some undetermined point in the data conversions to/from the server. Some whitespace (ascii code: oct 040, hex 0x20, dec 32) gets replaced with a different type of space (ascii code: oct 0137, hex 0x5f, dec 95). It doesn't happen for all spaces, only a few, so it's really odd. Reproduce by: import a file to a db, then back to a file, and compare the original file with the routed-through-mysql file. It happens on all data formats.

Db structure

CREATE TABLE nodeinfo (
    id int(11) NOT NULL auto_increment,
    parent_id int(11) NOT NULL default '0', // = parent node's nodeinfo.id, or 0
    class varchar(100) NOT NULL default '', // node_traits<>::class_name()
    name varchar(100) NOT NULL default '', // node_traits<>::name()
    PRIMARY KEY  (id)
) TYPE=MyISAM;

CREATE TABLE properties (
      node_id int(11) NOT NULL default '0', // = nodeinfo.id
      pkey varchar(255) NOT NULL default '', // property key
      pvalue text                            // property value
) TYPE=MyISAM;
CREATE TABLE pseudofiles (
      id int(11) NOT NULL auto_increment,   // "inode" number ;)
      node_id int(11) NOT NULL default '0', // = nodeinfo.id
      name varchar(255) NOT NULL default '', // "filename"
      PRIMARY KEY  (id)
) TYPE=MyISAM;

Using MyS with s11nconvert

Examples:

s11nconvert -dl mysql -f my.s11n -s mysql -o MyDbObject
// or, more simply:
s11nconvert -f my.s11n -s mysql -o MyDbObject

s11nconvert -S mysql -f MyDbObject -s simplexml > my.s11n

s11nconvert -S mysql -f MyDbObject -s mysql -o "My Copied Object"

s11nconvert -f in.paren -s mysql
#s11n::io::serializer mysql_serializer
<!DOCTYPE SerialTree>
<mysql_serializer_proxy_node class="s11n::data_node">
        <dbhost>localhost</dbhost>
        <dbname>mys11n</dbname>
        <dbpassword></dbpassword>
        <dbuser>s11n</dbuser>
        <node_id>6715</node_id>
</mysql_serializer_proxy_node>

s11nconvert -f infilename -s simplexml
<!DOCTYPE s11n::simplexml>
<somenode s11n_class="NoClass" a="()\b" foo="bar" long="this is a long property">
        <fred s11n_class="FredClass" key="value" />
        <wilma s11n_class="WilmaClass" the="lovely wife" />
        <betty s11n_class="BettyClass" value="the apple of Barney&apos;s eye" />
        <deep s11n_class="Foo">
                <deeper s11n_class="Foo" how_deep="really deep!">
                        <and_deeper s11n_class="Ouch" />
                </deeper>
        </deep>
</somenode>

Command-line options:

s11nconvert -dl mysql_serializer --help
<snip>
mysql-db:
        Sets the s11n/mysql database's name.
mysql-host:
        Sets the mysql db server.
mysql-password:
        Sets the mysql user's password.
mysql-user:
        Sets the mysql db user.
mysql-verbose:
        Toggles on verbose mode for mysql_serializer.
<snip>

Tips:

• It has an evil twin, -DL, which works identically but does not consider a failed DLL load to be an error (i.e., it ignores such errors). -dl, on the other hand, will fail if a given DLL cannot be loaded.
• It is optional if a library named SERALIZER.so or SERIALIZER_serializer.so can be found and opened, where SERIALIZER is the name passed to -S or -s.
• May be passed base-names if DLLNAME.so is in s11n's classpath (as described in the library manual).
• May be passed multiple times, unlike the other options. Use it once for each DLL you want to load, e.g. -dl MyDLL -dl YourDLL.
• s11nconvert does nothing with the DLLs except to open them. i.e., it does not look for any specific symbols in any DLLs. In the case of DLLs containing s11n registration code, this simple act is enough to activate the registrations they contain. You may, if you like, use this feature for opening non-s11n-related DLLs, but i can personally see little use in doing so in the context of s11nconvert.

Using MyS in client code

s11nlite::serializer_class( "mysql_serializer" );

// if serialize_class() has been set:
s11nlite::serializer_base_type * ser =
      s11nlite::create_serializer();
// otherwise:
s11nlite::serializer_base_type * ser =
      s11nlite::create_serializer( "mysql_serializer" );

#include <s11n.net/mysql_serializer/mysql_serializer.hpp>
#include <s11n.net/mysql_serializer/libsql++.hpp>
// If you want to see any error messages, you must
// register an error handler:
StderrLog errlog;
// Create db object and connect...
Database db(host,user,password,db,&errlog);
if( db.Connected() )
{
      delete( s11n::io::mysql_database() );
      s11n::io::mysql_database( &db );
}
else
{
      // bail out...
}