Developer forum

Re: [midgard-dev] RDF use-case

Piotr Pokora — Tue, 31 Aug 2010 15:00:06 +0000

Application developer can use aliased property-names/class-names, but internally system operates using full namespaced names. And when queries are prepared for StorageManager these names are mapped to SQL field names or whatever current storage requires.

So, we can have several StorageManagers but only one NamespaceManager.

Agree.

Now, that I think of it, other option is, to implement NamespaceManager on application level. It's quite easy to do and NamespaceManager doesn't provide any vital functionality. It is there only to make things easier for app-developers

It could be a wrapper for particular SchemaObject class.

Piotras

dev mailing list dev@lists.midgard-project.org http://lists.midgard-project.org/mailman/listinfo/dev

Re: [midgard-dev] RDF use-case

Alexey Zakhlestine — Tue, 31 Aug 2010 14:45:06 +0000

On 31.08.2010, at 17:46, Piotr Pokora wrote:

Why it can not associated with StorageManager which creates all storage related classes? To be consistent with Query, Model, Transaction and Profiler, Namespace manager should be created with StorageManager.

I don't like it either way.

NamespaceManager is not related to StorageManager at all. NamespaceManager is the property of the whole runtime context and is active on the level of SchemaModels

Application developer can use aliased property-names/class-names, but internally system operates using full namespaced names. And when queries are prepared for StorageManager these names are mapped to SQL field names or whatever current storage requires.

So, we can have several StorageManagers but only one NamespaceManager.

Now, that I think of it, other option is, to implement NamespaceManager on application level. It's quite easy to do and NamespaceManager doesn't provide any vital functionality. It is there only to make things easier for app-developers

dev mailing list dev@lists.midgard-project.org http://lists.midgard-project.org/mailman/listinfo/dev

Re: [midgard-dev] RDF use-case

Piotr Pokora — Tue, 31 Aug 2010 13:50:06 +0000

var nm = new NamespaceManager; // should it be singleton?

Yes. If we associate it with StorageManager:

singleton

lazy initialization

well the problem is, that NamespaceManager is not a property of storage though, NamespaceManager can (should?) use StorageManager as the backend

Maybe it should be associated with ModelManager, instead?:
var storage = new StorageManager(new Config());
storage.open();

var nm = storage.model_manager.namespace_manager;

Why it can not associated with StorageManager which creates all storage related classes? To be consistent with Query, Model, Transaction and Profiler, Namespace manager should be created with StorageManager.

On the other hand, if we would like to ModelManager to create Namespace one, why do not make Namespace manager yet another model?

Latter makes ModelManager implementation a bit difficult, and the first one forces us to clarify StorageManager family relations I mentioned few mails earlier.

Piotras

dev mailing list dev@lists.midgard-project.org http://lists.midgard-project.org/mailman/listinfo/dev

Re: [midgard-dev] RDF use-case

Alexey Zakhlestine — Tue, 31 Aug 2010 12:30:06 +0000

[[ 2-nd edition ]]

Vala code, for defining namespaces

var storage = new StorageManager(new Config()); storage.open();

var nm = storage.model_manager.namespace_manager;

nm.create_uri('http://purl.org/dc/elements/1.1/', 'dc'); nm.create_uri('http://example.com/client/', 'client');

Vala code for defining properties

var dc_title = new SchemaModelProperty(); dc_title.namespace = 'dc'; dc_title.name = 'title'; dc_title.value_typename = 'string';

var dc_date = new SchemaModelProperty(); dc_date.namespace = 'dc'; dc_date.name = 'date'; dc_date.value_typename = 'DateTime'; // from http://github.com/chergert/gdatetime

var dc_language = new SchemaModelProperty(); dc_language.namespace = 'dc'; dc_language.name = 'language'; dc_language.value_typename = 'string';

var client_book = new SchemaModelProperty(); client_book.namespace = 'client'; client_book.name = 'book'; client_book.value_typename = 'client:book'; // we'll need to define this beforehand

// ==================================================================== // We need means to store properties definitions at this point without // attaching them to classes
// ====================================================================

Vala code for defining classes

// http://example.com/client/book var book_model = new SchemaModel(); book_model.namespace = 'client'; book_model.name = 'book';

book_model .add_model(dc_title) .add_model(dc_date) .add_model(dc_language);

// http://example.com/client/movie var movie_model = new SchemaModel(); movie_model.namespace = 'client'; movie_model.name = 'movie';

movie_model .add_model(dc_title) .add_model(dc_date) .add_model(dc_language) .add_model(client_book);

// register in type-system var builder = new SchemaBuilder(); builder.register_model(book_model); builder.register_model(movie_model); builder.execute();

Vala code for creating objects of defined classes

var book = SchemaBuilder.factory("http://example.com/client/book") as SchemaObject; book.set_property_value('dc:title', 'Do Androids Dream of Electric Sheep?'); book.set_property_value('dc:date', '1968'); book.set_property_value('dc:language', 'en_EN');

var movie = SchemaBuilder.factory("http://example.com/client/movie") as SchemaObject; movie.set_property_value('dc:title', 'Blade_Runner'); movie.set_property_value('dc:date', '1982-06-25'); movie.set_property_value('dc:language', 'en_EN'); movie.set_property_value('client:book', book);

dev mailing list dev@lists.midgard-project.org http://lists.midgard-project.org/mailman/listinfo/dev

Re: [midgard-dev] RDF use-case

Alexey Zakhlestine — Tue, 31 Aug 2010 12:05:07 +0000

On 31.08.2010, at 15:09, Piotr Pokora wrote:

Vala code, for defining namespaces

var nm = new NamespaceManager; // should it be singleton?

Yes. If we associate it with StorageManager:

singleton

lazy initialization

well the problem is, that NamespaceManager is not a property of storage though, NamespaceManager can (should?) use StorageManager as the backend

Maybe it should be associated with ModelManager, instead?:

var storage = new StorageManager(new Config());
storage.open();

var nm = storage.model_manager.namespace_manager;

Vala code for defining properties

dc_title.name = 'title'; dc_title.value_gtype = gchararray;

I added typename and g_type. For typename, you can declare just 'string' which is common in every language, while gchararray is GType system specific. g_type let us tune schema vs storage models creation. For example datetime values which must be string ones in some DBMS.

ok. will post updated vala-example in a moment

Vala code for defining classes

// http://example.com/client/book var book_model = new SchemaModel(); book_model.namespace = 'client';

If class namespace is needed, please update Model interface.

done

dev mailing list dev@lists.midgard-project.org http://lists.midgard-project.org/mailman/listinfo/dev

Re: [midgard-dev] RDF use-case

Piotr Pokora — Tue, 31 Aug 2010 11:15:05 +0000

Vala code, for defining namespaces

var nm = new NamespaceManager; // should it be singleton?

Yes. If we associate it with StorageManager:

singleton
lazy initialization

Vala code for defining properties

dc_title.name = 'title'; dc_title.value_gtype = gchararray;

I added typename and g_type. For typename, you can declare just 'string' which is common in every language, while gchararray is GType system specific. g_type let us tune schema vs storage models creation. For example datetime values which must be string ones in some DBMS.

// ==================================================================== // We need means to store properties definitions at this point without // attaching them to classes
// ====================================================================

Proposal?

Vala code for defining classes

// http://example.com/client/book var book_model = new SchemaModel(); book_model.namespace = 'client';

If class namespace is needed, please update Model interface.

Piotras

dev mailing list dev@lists.midgard-project.org http://lists.midgard-project.org/mailman/listinfo/dev

Re: [midgard-dev] Midgard3 API proposal

Piotr Pokora — Tue, 31 Aug 2010 11:05:04 +0000

Do you mean this? MidgardStorage::setParameterAlias('HelloWorld', 'dc:title', 'title');

I think it can be managed via NamespaceManager.

I mean application developer should be able to use any of these if namespace and parameter-alias are defined:

obj.setParameter('http://dublincore.org/documents/dcmi-namespace/title', 'new_title'); obj.setParameter('dc:title', 'new_title'); obj.setParameter('title', 'new_title');

this, as the bonus, would be nice too:

obj.title = new_title

'new_title' is a value?

I added set(get)_property_value methods by purpose. If we set/get property, then any developer will look for property set/get method.

Another bonus: we leave deprecated API untouched.

Piotras

dev mailing list dev@lists.midgard-project.org http://lists.midgard-project.org/mailman/listinfo/dev

Re: [midgard-dev] RDF use-case

Piotr Pokora — Tue, 31 Aug 2010 10:55:10 +0000

namespaces: - uri: http://purl.org/dc/elements/1.1/ alias: dc - uri: http://example.com/client/ alias: client

properties: - uri: http://purl.org/dc/elements/1.1/title db_type: text # can be varchar(1024) for sql - uri: http://purl.org/dc/elements/1.1/date db_type: datetime - uri: http://purl.org/dc/elements/1.1/language db_type: text # can be varchar(50) for sql - uri: http://example.com/client/book db_type: uuid # reference to book. can be char(32) or binary(16) for sql

classes: - uri: http://example.com/client/book fields: - http://purl.org/dc/elements/1.1/title - http://purl.org/dc/elements/1.1/date - http://purl.org/dc/elements/1.1/language

- uri: http://example.com/client/movie fields: - http://purl.org/dc/elements/1.1/title - http://purl.org/dc/elements/1.1/date - http://purl.org/dc/elements/1.1/language - http://example.com/client/book ...

Vala code, for defining namespaces

With NamespaceManager in current shape:

NamespaceManager nsm = NamespaceManager (); nsm.create_uri ("http://purl.org/dc/elements/1.1/", "dc"); nsm.create_uri ("http://example.com/client/", "client"); nsm.create_uri ("uri: http://purl.org/dc/elements/1.1/title", "title");

Latter case clearly shows there's additional reference needed.

Vala code for defining properties

SchemaModelProperty title_p = SchemaModelProperty ("title", "string", "", "The title"); SchemaModelProperty date_p = SchemaModelProperty ("date", "datetime", "0000-01-01 00:00:00", "Date"); SchemaModelProperty language_p = SchemaModelProperty ("language", "string", "", "Languge");

Vala code for defining classes

SchemaModel book_model = SchemaModel ("Book"); SchemaModel movie_model = SchemaModel ("Movie");

book_model.add_model (title_p).add_model (date_p).add_model (language_p);

movie_model.add_model(title_p).add_model (date_p).add_model (language_p).add_model (book_model);

SchemaBuilder schema_builder = SchemaBuilder ();

try { schema_builder.register_model (book_model); schema_builder.register_model (movie_model); schema_builder.execute (); } catch () { }

Vala code for creating objects of defined classes

StorageModelManager model_manager = storage.get_model_manager();

StorageModel book_storage_model = model_manager.create_storage_model (book_model, "books_table"); StorageModel movie_storage_model = model_manager.create_storage_model (movie_model, "movies_table");

StorageModelProperty title_storage = StorageModelProperty (title_p, "title_field"); StorageModelProperty date_storage = StorageModelProperty (date_p, "date_field"); StorageModelProperty lang_storage = StorageModelProperty (language_p, "lang_field");

book_storage_model.add_model (title_storage).add_model (date_storage).add_model (lang_storage);

StorageModelProperty book_storage = StorageModelProperty (book_model, "book_field");

movie_storage_model.add_model (title_storage).add_model (date_storage).add_model (lang_storage).add_model (book_storage);

model_manager.add_model (book_storage_model).add_model (book_model).add_model (movie_storage_model).add_model (movie_model);

try { model_manager.prepare_create (); model_manager.execute (); } catch () {}

SchemaObject book = schema_builder.factory ("Book"); book.set_property_value ("http://purl.org/dc/elements/1.1/title", "The true story of the movie");

ContentManager cm = storage.get_content_manager(); cm.create (book);

SchemaObject movie = schema_builder.factory ("Movie"); book.set_property_value ("http://purl.org/dc/elements/1.1/title", "The great movie"); book.set_property_value ("http://example.com/client/book", book);

cm.create (movie);

Piotras

dev mailing list dev@lists.midgard-project.org http://lists.midgard-project.org/mailman/listinfo/dev

Re: [midgard-dev] RDF use-case

Alexey Zakhlestine — Tue, 31 Aug 2010 10:55:07 +0000

On 31.08.2010, at 13:49, Alexey Zakhlestin wrote:

Vala code, for defining namespaces

var nm = new NamespaceManager; // should it be singleton?

nm.create_uri('http://purl.org/dc/elements/1.1/', 'dc'); nm.create_uri('http://example.com/client/', 'client');

Vala code for defining properties

var dc_title = new SchemaModelProperty(); dc_title.namespace = 'dc'; dc_title.name = 'title'; dc_title.value_gtype = gchararray;

var dc_date = new SchemaModelProperty(); dc_date.namespace = 'dc'; dc_date.name = 'date'; dc_date.value_gtype = GDateTime; // from http://github.com/chergert/gdatetime

var dc_language = new SchemaModelProperty(); dc_language.namespace = 'dc'; dc_language.name = 'language'; dc_language.value_gtype = gchararray;

var client_book = new SchemaModelProperty(); client_book.namespace = 'client'; client_book.name = 'book'; client_book.value_gtype = ClientBook; // we'll need to define this beforehand

Vala code for defining classes

// http://example.com/client/book var book_model = new SchemaModel(); book_model.namespace = 'client'; book_model.name = 'book';

book_model .add_model(dc_title) .add_model(dc_date) .add_model(dc_language);

// http://example.com/client/movie var movie_model = new SchemaModel(); movie_model.namespace = 'client'; movie_model.name = 'movie';

movie_model .add_model(dc_title) .add_model(dc_date) .add_model(dc_language) .add_model(client_book);

// register in type-system var builder = new SchemaBuilder(); builder.register_model(book_model); builder.register_model(movie_model); builder.execute();

Vala code for creating objects of defined classes

dev mailing list dev@lists.midgard-project.org http://lists.midgard-project.org/mailman/listinfo/dev

Re: [midgard-dev] Midgard3 API proposal

Piotr Pokora — Tue, 31 Aug 2010 10:05:07 +0000

There are two more issues (at least :) I didn't resolve so far. StorageManager is a (kind of) facade for other storage related classes. Like ContentManager, ModelManager or QueryManager.

Returned QueryManager could be responsible to create particular Query class, like Select, Iterator, Data, and so on. I wonder if it makes sense to provide such functionality (with predefined and limited number of Query classes) or make them standalone, constructed for given StorageManager?

well, my thoughts were, that exact number of storage-query classes is dependent on storage.

Yes, thinking about this too.

we just need to have: 1) some entry-point, which will be able to convert schema-query to storage-query (and how exactly this storage-query looks like is just "implementation detail" which is not important to application developer)

Can you describe what is schema-query?

2) some entry point, which will be able to execute this storage-query (that will be Executable interface, I guess)

as long as these are present Storage can have any set of query-classes (in the edge-case it even can be single object holding sql-string)

am I talking nonsense? ;-)

No. I added QueryManager by purpose. This one should be responsible to create proper Query object of the storage it belongs too. Tell you what Query tools are available, etc.

Another issue is similar. I am pretty sure, with SQL StorageManager implementation, we will need more than one StorageManager (different SQL class instance) objects sharing the same underlying database connection. Those might be, for example, raw SQL and Workspace managers used by the same application.

well we can represent it as the system of pipes/transformers, which convert queries to other form.

Something like this:

SchemaQuery => SchemaQuery in Workspace => StorageQuery ^ ^ (WorkspaceMapper) (StorageMapper)

in this case, we don't need to have actual connection for the workspace. workspace can operate on higher level, above any storage-details.

I am thinking about different thing. I am thing about possibility to reuse very low level (from API point of view) and internal resources, like database connection or file handler (if needed).

Possible scenario (let's use SQL storage):

I need to authenticate user, so I use raw SQL query.
User creates some document which should be stored in Workspace A, so I use Workspace content manager
User gets some document, which uses RDF ontology and should be updated, so I use SPARQL content manager.

If those three content managers, need to use the same database, should application create three different StorageManagers and thus, three different database connections?

Main idea is that any object is storage unaware, so any query or content manager is responsible to store and query data its own way.

With current state of the API, there should be three storage managers initialized for this case:

StorageManagerSQL
StorageManagaerSQLWorkspace
StorageManagerSPARQL

This is bad from resources usage point of view. But it gives us clean runtime solution.

Piotras

dev mailing list dev@lists.midgard-project.org http://lists.midgard-project.org/mailman/listinfo/dev