Modern programming languages have concept of namespaces within them. Midgard, historically, used underscores as pseudo-namespacing tool, but these days we try to use real namespaces in python (and in php soon). Using namespaces in python led to the following code:
obj = midgard.mgdschema.midgard_article()
This code mentions "midgard" 2 times which doesn't look nice. Direct approach to fixing this leads to bad hacks and breaks backwards compatibility. See bug #1559 as an example. It removes midgard_ prefix from all type-names, which can result in names-conflict if there is some type which had the unprefixed name. Clearly a fully designed approach is needed because the API can't change in every Midgard release and we can't support multiple legacy syntaxes.
This document is a proposal for discussion on implementation, which would work as the long-term solution for all future language bindings.
We propose to add explicit <namespace> tag in schema declaration file. Namespaces are supposed to have <class> tags as their children. On the side of language binding it might be translated either to true namespaces (python, php-5.3, etc.) or in pseudo-namespaces (php-5.2) depending on what binding author want/can do. For namespace-capable bindings we propose to use midgard.schema as the root namespace for everything.
It is recommended to prefix table names with namespace_. It shouldn't be enforced, but as a recommended practice would prevent table name conflicts and ease database-management (all tables would be grouped by component, if listed alphabetically). "table" attribute of <class> tag is optional (if left out, table name would be namespace_class) and it is recommended to leave it unset (less coupling between code and actual database implementation).
For backwards compatibility reasons, old syntax with missing namespace tags and <type> tags should still be supported. In this case class should be put in midgard.mgdschema (note that it is mgdschema, not schema as in first clause) namespace. This is a strictly backwards-compatibility option. Such files must not be distributed with Ratatoskr.
<namespace name="core">
<class name="host" table="core_host">
…
</class>
…
</namespace>
<namespace name="my_component">
<class name="host" table="my_component_host">
…
</class>
…
</namespace>
And compatibility mode:
<!-- Compatibility example -->
<type name="midgard_host" table="host>
…
</type>
<type name="my_component_host" table="my_component_host">
…
</type>
<!-- /Compatibility example -->
obj = midgard.schema.core.host()
obj2 = midgard.schema.my_component.host()
# we can use class-name directly, instead of string. python allows it
qb = midgard.query_builder(midgard.schema.my_component.host)
# this should also work
from midgard.schema import core
from midgard.schema.my_component import *
obj = core.host()
obj2 = host()
qb = midgard.query_builder(core.host)
qb2 = midgard.query_builder(host)
and for "compatibility" mode:
obj = midgard.mgdschema.midgard_host()
obj2 = midgard.mgdschema.my_component_host()
qb = midgard.query_builder(midgard.mgdschema.my_component_host)
# string-parameter should work for "old schemas" in ratatoskr too (with deprecated-notice)
# (strings translate to classes under the midgard.mgdschema namespace only)
qb = midgard.query_builder('my_component_host')
$obj = new midgard\schema\core\host();
$obj2 = new midgard\schema\my_component\host();
// in php we can't pass class as parameter. have to use string.
// (use single quotes because with double quotes e.g. \n wouldn't work like expected)
$qb = new midgard\query_builder('midgard\schema\my_component\host');
// but this should work
$qb = midgard\schema\my_component\host::new_query_builder();
// and this
use midgard\schema\my_component as my;
$qb = my\host::new_query_builder();
and for "compatibility" mode:
$obj = new midgard_host();
$obj2 = new my_component_host();
$qb = new midgard_query_builder('my_component_host');
$qb2 = my_component_host::new_query_builder();
if we continue to support php 5.2…
// "core" is represented as "midgard_" for historical reasons
$obj = new midgard_host();
$obj2 = new my_component_host();
$qb = new midgard_query_builder('my_component_host');
$qb2 = my_component_host::new_query_builder();
// "compatibility" mode translates to same API as above
Modern programming languages have concept of namespaces within them. Midgard, historically, used underscores as pseudo-namespacing tool, but these days we try to use real namespaces in python (and in php soon). Using namespaces in python led to the following code:
obj = midgard.mgdschema.midgard_article()
This code mentions "midgard" 2 times which doesn't look nice. Direct approach to fixing this leads to bad hacks and breaks backwards compatibility. See bug #1559 as an example. It removes midgard_ prefix from all type-names, which can result in names-conflict if there is some type which had the unprefixed name. Clearly a fully designed approach is needed because the API can't change in every Midgard release and we can't support multiple legacy syntaxes.
This document is a proposal for discussion on implementation, which would work as the long-term solution for all future language bindings.
We propose to add explicit <namespace> tag in schema declaration file. Namespaces are supposed to have <class> tags as their children. On the side of language binding it might be translated either to true namespaces (python, php-5.3, etc.) or in pseudo-namespaces (php-5.2) depending on what binding author want/can do. For namespace-capable bindings we propose to use midgard.schema as the root namespace for everything.
It is recommended to prefix table names with namespace_. It shouldn't be enforced, but as a recommended practice would prevent table name conflicts and ease database-management (all tables would be grouped by component, if listed alphabetically). "table" attribute of <class> tag is optional (if left out, table name would be namespace_class) and it is recommended to leave it unset (less coupling between code and actual database implementation).
For backwards compatibility reasons, old syntax with missing namespace tags and <type> tags should still be supported. In this case class should be put in midgard.mgdschema (note that it is mgdschema, not schema as in first clause) namespace. This is a strictly backwards-compatibility option. Such files must not be distributed with Ratatoskr.
<namespace name="core">
<class name="host" table="core_host">
…
</class>
…
</namespace>
<namespace name="my_component">
<class name="host" table="my_component_host">
…
</class>
…
</namespace>
And compatibility mode:
<!-- Compatibility example -->
<type name="midgard_host" table="host>
…
</type>
<type name="my_component_host" table="my_component_host">
…
</type>
<!-- /Compatibility example -->
obj = midgard.schema.core.host()
obj2 = midgard.schema.my_component.host()
# we can use class-name directly, instead of string. python allows it
qb = midgard.query_builder(midgard.schema.my_component.host)
# this should also work
from midgard.schema import core
from midgard.schema.my_component import *
obj = core.host()
obj2 = host()
qb = midgard.query_builder(core.host)
qb2 = midgard.query_builder(host)
and for "compatibility" mode:
obj = midgard.mgdschema.midgard_host()
obj2 = midgard.mgdschema.my_component_host()
qb = midgard.query_builder(midgard.mgdschema.my_component_host)
# string-parameter should work for "old schemas" in ratatoskr too (with deprecated-notice)
# (strings translate to classes under the midgard.mgdschema namespace only)
qb = midgard.query_builder('my_component_host')
$obj = new midgard\schema\core\host();
$obj2 = new midgard\schema\my_component\host();
// in php we can't pass class as parameter. have to use string.
// (use single quotes because with double quotes e.g. \n wouldn't work like expected)
$qb = new midgard\query_builder('midgard\schema\my_component\host');
// but this should work
$qb = midgard\schema\my_component\host::new_query_builder();
// and this
use midgard\schema\my_component as my;
$qb = my\host::new_query_builder();
and for "compatibility" mode:
$obj = new midgard_host();
$obj2 = new my_component_host();
$qb = new midgard_query_builder('my_component_host');
$qb2 = my_component_host::new_query_builder();
if we continue to support php 5.2…
// "core" is represented as "midgard_" for historical reasons
$obj = new midgard_host();
$obj2 = new my_component_host();
$qb = new midgard_query_builder('my_component_host');
$qb2 = my_component_host::new_query_builder();
// "compatibility" mode translates to same API as above
On June 30, 2009 PHP 5.3.0 was released. It brings major functionality changes, which would bring a lot of benefits for Midgard-MVC. Midgard-MVC is, currently, a part of Mjölnir release. Mjölnir is not planned to be a LTS-release, but as a first release of midgard-2 for building produciton sites upon.
I propose to switch Mjölnir development to PHP-5.3+ only compatibility. As a result we will get a cleaner, faster and more powerful code. It would be a bad idea to break compatibility after we release Mjölnir. Majority of application-developers wouldn't rely on Mjölnir for quite some time, which gives unix-distributions enough time to include PHP-5.3 in their releases.
In 5.2 and earlier callbacks could be either runtime-compiled (create_function()), which is slow, or you had to define method and pass it's name, which clutters namespace. Anyway, both these cases lack scope-inheritance which limits their usefulness.
5.3 has true closures, which allows usage of functional patterns in php code. First thing which comes to mind is work with arrays: map, reduce, filter, etc. instead of foreach-cycles. But even more interesting pattern is context-aware callbacks. There's a nice example of such callback here
function loadData($url, $handler)
{
$handler(file_get_contents($url));
}
function sendPageTo($email, $url)
{
loadData($url, function($data) use ($email) {
mail($email, "Here's your data", $data);
});
}
In general, closures give us a whole new dimension of reusability.
This, probably, is the most discussed feature of PHP 5.3. I'll be short: we have use-case for this. Current naming pattern in Midgard-MVC leads to extremely long class-names. Switching to namespaces would make code more compact, more readable and less fragile.
Also, there is an interesting related topic: using exception-model proposed by php.standards initiative. See "PHP Standards and Best Practices for PHP 5.3+ Frameworks and Libraries" document.
Basic idea is, that application should inherit it's exceptions from the standard SPL-exceptions while giving them the same name (but in it's own namespaces)
namespace Midgard;
class RuntimeException extends \RuntimeException();
after that, framework-developers can use usual "throw new RuntimeException();" in code, and application developers can distinguish between exceptions which come from different frameworks:
try {
// some code
} catch (Midgard\RuntimeException $e) {
// Midgard-specific handling
} catch (PEAR2\RuntimeException $e) {
// PEAR-specific handling
} catch (RuntimeException $e) {
// Generic handling
}
This is a major step ahead in PHP's dynamic nature. My favourite example for this feature is ActiveRecord-like classes. In PHP 5.2 it was possible to define ActiveRecord class with generic methods for loading, creating, updating of database-records. After that, it was possible to inherit another class from it, while not defining class body. Like this, literally:
class Book extends ActiveRecord {}
$book = new Book();
$book->name = 'name';
$book->author = 'author';
$book->save();
ActiveRecord's save() method, in this case, would detect object's class at runtime and figure out what to do dynamically. That's a common knowledge and is really useful. But… It was impossible to do the same with class-methods (aka "static" methods) until now. Class methods couldn't detect the class they were called on. So, in 5.2 it was impossible to implement generic method for the following case:
$books = Book::find_all();
ActiveRecord's find_all() method would only know, that it belongs to ActiveRecord and wouldn't know anything about Book class.
PHP 5.3 introduces 2 things in this regard:
check PHP Manual for more details
This one is simple. In PHP 5.2 and earlier, if you had situation where object $a has reference to object $b, while, at the same time, $b has reference to $a the memory they occupy would never be freed (until program termination). This limitation, effectively, killed possibility of having complex long-running php processes. Sooner or later, process would take all available memory and die("in agony").
PHP 5.3 fixes this and gives us ability to write complex daemons in php.
Starting with PHP 5.3, SPL (Standard PHP Library) gains real value. Until now it was a set of small, helpful, but mostly under-the-hood goodies. 5.3 brings fast implementations of the most common data-structures:
Usual speed difference between this implementation and implementation in PHP is several tens times, at least. They are really fast.
(spell-checking), intl (unicode-compliant collation + ICU-based formatting)
PHAR let's us pack a library or an application into a single "project.phar" file. That is convenient for distribution and gives speed-boost if you use phar and APC at the same time (APC can use more efficient cache-strategy, when application is packed as a single-file on filesystem level).
Enchant is a generalized extension, which can work with the majority of spell-checking backends. This includes ispell, aspell and even AppleSpell
intl (aka Internationalization extension) is a binding to ICU (International Components for Unicode) library. It provides means for working with unicode-compliant locale's, proper unicode-compliant sorting and unicode-compliant formatting of dates and numbers. That's a necessity in 21-st century and POSIX-locales are way outdated to handle complex cases properly.
]]>On June 30, 2009 PHP 5.3.0 was released. It brings major functionality changes, which would bring a lot of benefits for Midgard-MVC. Midgard-MVC is, currently, a part of Mjölnir release. Mjölnir is not planned to be a LTS-release, but as a first release of midgard-2 for building produciton sites upon.
I propose to switch Mjölnir development to PHP-5.3+ only compatibility. As a result we will get a cleaner, faster and more powerful code. It would be a bad idea to break compatibility after we release Mjölnir. Majority of application-developers wouldn't rely on Mjölnir for quite some time, which gives unix-distributions enough time to include PHP-5.3 in their releases.
In 5.2 and earlier callbacks could be either runtime-compiled (create_function()), which is slow, or you had to define method and pass it's name, which clutters namespace. Anyway, both these cases lack scope-inheritance which limits their usefulness.
5.3 has true closures, which allows usage of functional patterns in php code. First thing which comes to mind is work with arrays: map, reduce, filter, etc. instead of foreach-cycles. But even more interesting pattern is context-aware callbacks. There's a nice example of such callback here
function loadData($url, $handler)
{
$handler(file_get_contents($url));
}
function sendPageTo($email, $url)
{
loadData($url, function($data) use ($email) {
mail($email, "Here's your data", $data);
});
}
In general, closures give us a whole new dimension of reusability.
This, probably, is the most discussed feature of PHP 5.3. I'll be short: we have use-case for this. Current naming pattern in Midgard-MVC leads to extremely long class-names. Switching to namespaces would make code more compact, more readable and less fragile.
Also, there is an interesting related topic: using exception-model proposed by php.standards initiative. See "PHP Standards and Best Practices for PHP 5.3+ Frameworks and Libraries" document.
Basic idea is, that application should inherit it's exceptions from the standard SPL-exceptions while giving them the same name (but in it's own namespaces)
namespace Midgard;
class RuntimeException extends \RuntimeException();
after that, framework-developers can use usual "throw new RuntimeException();" in code, and application developers can distinguish between exceptions which come from different frameworks:
try {
// some code
} catch (Midgard\RuntimeException $e) {
// Midgard-specific handling
} catch (PEAR2\RuntimeException $e) {
// PEAR-specific handling
} catch (RuntimeException $e) {
// Generic handling
}
This is a major step ahead in PHP's dynamic nature. My favourite example for this feature is ActiveRecord-like classes. In PHP 5.2 it was possible to define ActiveRecord class with generic methods for loading, creating, updating of database-records. After that, it was possible to inherit another class from it, while not defining class body. Like this, literally:
class Book extends ActiveRecord {}
$book = new Book();
$book->name = 'name';
$book->author = 'author';
$book->save();
ActiveRecord's save() method, in this case, would detect object's class at runtime and figure out what to do dynamically. That's a common knowledge and is really useful. But… It was impossible to do the same with class-methods (aka "static" methods) until now. Class methods couldn't detect the class they were called on. So, in 5.2 it was impossible to implement generic method for the following case:
$books = Book::find_all();
ActiveRecord's find_all() method would only know, that it belongs to ActiveRecord and wouldn't know anything about Book class.
PHP 5.3 introduces 2 things in this regard:
check PHP Manual for more details
This one is simple. In PHP 5.2 and earlier, if you had situation where object $a has reference to object $b, while, at the same time, $b has reference to $a the memory they occupy would never be freed (until program termination). This limitation, effectively, killed possibility of having complex long-running php processes. Sooner or later, process would take all available memory and die("in agony").
PHP 5.3 fixes this and gives us ability to write complex daemons in php.
Starting with PHP 5.3, SPL (Standard PHP Library) gains real value. Until now it was a set of small, helpful, but mostly under-the-hood goodies. 5.3 brings fast implementations of the most common data-structures:
Usual speed difference between this implementation and implementation in PHP is several tens times, at least. They are really fast.
(spell-checking), intl (unicode-compliant collation + ICU-based formatting)
PHAR let's us pack a library or an application into a single "project.phar" file. That is convenient for distribution and gives speed-boost if you use phar and APC at the same time (APC can use more efficient cache-strategy, when application is packed as a single-file on filesystem level).
Enchant is a generalized extension, which can work with the majority of spell-checking backends. This includes ispell, aspell and even AppleSpell
intl (aka Internationalization extension) is a binding to ICU (International Components for Unicode) library. It provides means for working with unicode-compliant locale's, proper unicode-compliant sorting and unicode-compliant formatting of dates and numbers. That's a necessity in 21-st century and POSIX-locales are way outdated to handle complex cases properly.
]]>This is related to the problem mRFC 0046 tries to address but will propose approach that is usable for the older generations as well (for various reasons it's practical impossibility to require unit tests in pre-Vinland MidCOM).
In the dev list
See this discussion for background and details.
This is related to the problem mRFC 0046 tries to address but will propose approach that is usable for the older generations as well (for various reasons it's practical impossibility to require unit tests in pre-Vinland MidCOM).
In the dev list
See this discussion for background and details.
Nowdays anyone can commit to the repo of ragnaroek and to the new midcom 3. I'm not talking about how things work in MidCOM 3, because so few is using or commiting to the yet. So, the examples I'm writing are from ragnaroek and the main idea of this mRFC is to avoid these problems in MidCOM 3 (I speak of MidCOM 3, even though we desided to ditch that name and start using Midgard CMS instead).
Problems that I have had lately with the midcom have allways somewhat related to something that was working being broken all of a sudden. And the only thing that I did, was to upgrade some midcom-component, because some other fix was made to the new release. So, while something is fixed, the new version allways brings other new 'features' along, that somehow broke things. This is because everyone has commit rights to the repo and no one has enought time to test the every outcome of their changes to the code. So, we need some rules for commiting.
The codebase of midcom being so big as it is now, there is no possibility for anyone to see all the possible risks in a new feature all by himself, at least when that someone needs to work at the same time. So, what I propose, is to have testing and stable branches and make some sort of a valuation when moving fixes and features from testing-branch to the stable branch. Now we have the trunk for ragnaroek, whitch automatically becomes 'stable', when a new release is made. So, there should be selected group of people, who can approve a set of commits from the testing-branch to the stable one.
For any feature needed to be in the stable branch, there is someone who needs it there. So, he / she makes a document containing following things
After the person neading the new features in the stable branch has made this request with enought test data, those who can approve commits from testing to stable, make the changes. So, for those who can approve, the job really isn't that big, because the coder has done all the testing and made a report about it. Report can be then made public, so afterwards others can see what happened and where.
As a side note at this point, those who are able to approve commits, can't approve their own commits. They need allways some other to approve their commits.
Other option is to have someone, who really has time to run tests after every commit. But as we don't have the Midgard Foundation yet, I see no possibility in here. And to be honest, testing others commits might be a bit hard job.
Basicly, what I'm saying, is that the current situation is unbearable and we need to do things a bit differently than it is now.
]]>Nowdays anyone can commit to the repo of ragnaroek and to the new midcom 3. I'm not talking about how things work in MidCOM 3, because so few is using or commiting to the yet. So, the examples I'm writing are from ragnaroek and the main idea of this mRFC is to avoid these problems in MidCOM 3 (I speak of MidCOM 3, even though we desided to ditch that name and start using Midgard CMS instead).
Problems that I have had lately with the midcom have allways somewhat related to something that was working being broken all of a sudden. And the only thing that I did, was to upgrade some midcom-component, because some other fix was made to the new release. So, while something is fixed, the new version allways brings other new 'features' along, that somehow broke things. This is because everyone has commit rights to the repo and no one has enought time to test the every outcome of their changes to the code. So, we need some rules for commiting.
The codebase of midcom being so big as it is now, there is no possibility for anyone to see all the possible risks in a new feature all by himself, at least when that someone needs to work at the same time. So, what I propose, is to have testing and stable branches and make some sort of a valuation when moving fixes and features from testing-branch to the stable branch. Now we have the trunk for ragnaroek, whitch automatically becomes 'stable', when a new release is made. So, there should be selected group of people, who can approve a set of commits from the testing-branch to the stable one.
For any feature needed to be in the stable branch, there is someone who needs it there. So, he / she makes a document containing following things
After the person neading the new features in the stable branch has made this request with enought test data, those who can approve commits from testing to stable, make the changes. So, for those who can approve, the job really isn't that big, because the coder has done all the testing and made a report about it. Report can be then made public, so afterwards others can see what happened and where.
As a side note at this point, those who are able to approve commits, can't approve their own commits. They need allways some other to approve their commits.
Other option is to have someone, who really has time to run tests after every commit. But as we don't have the Midgard Foundation yet, I see no possibility in here. And to be honest, testing others commits might be a bit hard job.
Basicly, what I'm saying, is that the current situation is unbearable and we need to do things a bit differently than it is now.
]]>Current Midgard's authentication system uses several hardcoded scenarios and doesn't reflect needs of modern web-world. This mRFC focuses mostly on implementation part, as it's impossible to describe idea for any single authentication mechanism.
Midgard needs modular authentication system which would allow developers to implement various authentication schemes for their needs. Midgard core shall provide abstraction layer API for any authentication type provided on any level and written with any language for which Midgard core provides bindings. Any new authentication type shall not require additional routines to be written in core or on language bindings level.
It should be possible to use it for implementing at least the following:
Basic idea is to provide routines which would be very similar to already known Midgard "Trusted" auth type. Midgard core shall implement midgard_auth and midgard_user classes.
guid/uuid - read only object's identifier guid/uuid property should be unique per table and mandatory for proper authentication.
login - string which identifies user (username/nick) non unique string with allowed empty one
password - stored in text/blob column so any data can be written to non unique string with allowed empty one
person - guid/uuid, a reference to midgard_person object non unique guid or uuid
is_admin - admin flag boolean value which marks user as admin
type - user defined field which holds integer value
If more properties are needed, a new MgdSchema class which extends midgard_user should be created.
array query(array properties) Fetch midgard_user object(s) using given properties as query constraints. If all properties match, object(s) is(are) returned from database.
Example, get all particular user accounts:
$tokens = array("login" => "john", "type" => MYAPP_AUTH_TYPE_CONST); $objects = midgard_user::query($tokens);
if (empty($objects)) print "No account for user 'john'";
boolean register(void) Creates new midgard_user objects' record in database.
boolean update(void) Updates midgard_user object's record in database.
Example:
$user = new midgard_user(); $user->login = "john"; $user->password = "";
if ($user->register()) print "Created new account for user 'john'";
midgard_person get_person(void); Returns midgard_person object associated (if any) with midgard_user instance.
boolean login(void) Login shall associate midgard_user object with midgard_connection, which shall emit 'auth-changed' signal and set proper access error code. User without guid set shall be considered invalid, 'auth-changed' signal shall not be emited and proper access error code shall be set. Only midgard_connection shall be responsible for setting error codes and signal emission.
boolean logout(void) Logout shall unset already associated midgard_user object. midgard_connection shall emit 'auth-changed' signal if associated with any midgard_user. No error code shall be set by midgard_connection.
By default midgard_user should not be accesible with query builder, however if new MgdSchema class which extends midgard_user is defined, it should be queryable with query builder or collector.
Authentication should be done with 'login' and 'logout' methods of midgard_user class. Midgard core will accept any midgard_user which logs in as long as it's fetched from database. It's up to the application if given midgard_user is valid or not. Read only guid/uuid property will be always set internally by core and used as "fetched from database" marker.
Once object is fetched from database, it can be stored (serialized) in file or as memcached key for later reuse. midgard_connection's get_user() method will provide pointer to actually logged in user.
midgard_connection shall hold authenticated users in a stack. Every successful login method invoked by midgard_user object shall add new user to the top of the stack, while its logout method shall remove user from the top of the stack.
midgard_user can be removed from top of the stack in two cases:
In latter case, Midgard core will invoke logout method in user's destructor.
It's application specific to logout users in proper order. Midgard core will return false for logout method, and will throw unconditional warning if there's attempt to logout a user which hasn't been logged in recently (it's not on top of the stack). Midgard language bindings should raise exception in such case, if it's only possible.
It's up to the application if user holds a reference to midgard_person object. If current user doesn't have a reference to midgard_person object (via guid or uuid), core won't raise any warning or error. In such case empty string will be used instead of person's identifier for MgdSchema objects' metadata properties like creator or revisor.
Legacy authentication will be implemented as class which extends midgard_user. To avoid bulk updates of legacy accounts, this class will register any account using midgard_user routines on the fly. Which means, new user account will be created (for legacy one) as soon as first authentication attempt is made. This could also clear username and password from midgard_person record transparently.
Implementation shall not allow any midgard_auth derived class to login in as Midgard root user or into sitegroup 0.
]]>Current Midgard's authentication system uses several hardcoded scenarios and doesn't reflect needs of modern web-world. This mRFC focuses mostly on implementation part, as it's impossible to describe idea for any single authentication mechanism.
Midgard needs modular authentication system which would allow developers to implement various authentication schemes for their needs. Midgard core shall provide abstraction layer API for any authentication type provided on any level and written with any language for which Midgard core provides bindings. Any new authentication type shall not require additional routines to be written in core or on language bindings level.
It should be possible to use it for implementing at least the following:
Basic idea is to provide routines which would be very similar to already known Midgard "Trusted" auth type. Midgard core shall implement midgard_auth and midgard_user classes.
guid/uuid - read only object's identifier guid/uuid property should be unique per table and mandatory for proper authentication.
login - string which identifies user (username/nick) non unique string with allowed empty one
password - stored in text/blob column so any data can be written to non unique string with allowed empty one
person - guid/uuid, a reference to midgard_person object non unique guid or uuid
is_admin - admin flag boolean value which marks user as admin
type - user defined field which holds integer value
If more properties are needed, a new MgdSchema class which extends midgard_user should be created.
array query(array properties) Fetch midgard_user object(s) using given properties as query constraints. If all properties match, object(s) is(are) returned from database.
Example, get all particular user accounts:
$tokens = array("login" => "john", "type" => MYAPP_AUTH_TYPE_CONST); $objects = midgard_user::query($tokens);
if (empty($objects)) print "No account for user 'john'";
boolean register(void) Creates new midgard_user objects' record in database.
boolean update(void) Updates midgard_user object's record in database.
Example:
$user = new midgard_user(); $user->login = "john"; $user->password = "";
if ($user->register()) print "Created new account for user 'john'";
midgard_person get_person(void); Returns midgard_person object associated (if any) with midgard_user instance.
boolean login(void) Login shall associate midgard_user object with midgard_connection, which shall emit 'auth-changed' signal and set proper access error code. User without guid set shall be considered invalid, 'auth-changed' signal shall not be emited and proper access error code shall be set. Only midgard_connection shall be responsible for setting error codes and signal emission.
boolean logout(void) Logout shall unset already associated midgard_user object. midgard_connection shall emit 'auth-changed' signal if associated with any midgard_user. No error code shall be set by midgard_connection.
By default midgard_user should not be accesible with query builder, however if new MgdSchema class which extends midgard_user is defined, it should be queryable with query builder or collector.
Authentication should be done with 'login' and 'logout' methods of midgard_user class. Midgard core will accept any midgard_user which logs in as long as it's fetched from database. It's up to the application if given midgard_user is valid or not. Read only guid/uuid property will be always set internally by core and used as "fetched from database" marker.
Once object is fetched from database, it can be stored (serialized) in file or as memcached key for later reuse. midgard_connection's get_user() method will provide pointer to actually logged in user.
midgard_connection shall hold authenticated users in a stack. Every successful login method invoked by midgard_user object shall add new user to the top of the stack, while its logout method shall remove user from the top of the stack.
midgard_user can be removed from top of the stack in two cases:
In latter case, Midgard core will invoke logout method in user's destructor.
It's application specific to logout users in proper order. Midgard core will return false for logout method, and will throw unconditional warning if there's attempt to logout a user which hasn't been logged in recently (it's not on top of the stack). Midgard language bindings should raise exception in such case, if it's only possible.
It's up to the application if user holds a reference to midgard_person object. If current user doesn't have a reference to midgard_person object (via guid or uuid), core won't raise any warning or error. In such case empty string will be used instead of person's identifier for MgdSchema objects' metadata properties like creator or revisor.
Legacy authentication will be implemented as class which extends midgard_user. To avoid bulk updates of legacy accounts, this class will register any account using midgard_user routines on the fly. Which means, new user account will be created (for legacy one) as soon as first authentication attempt is made. This could also clear username and password from midgard_person record transparently.
Implementation shall not allow any midgard_auth derived class to login in as Midgard root user or into sitegroup 0.
]]>The idea is to allow component developers specify more fine-grained access-control that works on gObject level, these can be set per property and both on the MgdSchema XML file and on API level.
Whether property can be read, can be used for write-only properties on mgdschema level (passwords etc) or to restict access to private data when access to the whole object should not be restricted (from example disallow read of event title but allow read of the dates)
Whether property can be written to, if user does not have write privileges for the object we can flag all properties as not writable and get extra layer of safety.
On MgdSchema two new parameters to the property are added "read" and "write", both default to "true".
Here we can change a true to false but not vice versa, ie we can add additional restrictions but not remove existing ones, these are added per object instance, for example using the following syntax:
$object->add_acl('property', 'write', false);
Trying to change an existing ACL setting from false to true will throw an exception.
We get signal of an 'event' object being instantiated and we see that this event has 'private' property set to true but the current user is not the creator of the event, so we deny read on event title and some other properties we consider to containt private data.
]]>The idea is to allow component developers specify more fine-grained access-control that works on gObject level, these can be set per property and both on the MgdSchema XML file and on API level.
Whether property can be read, can be used for write-only properties on mgdschema level (passwords etc) or to restict access to private data when access to the whole object should not be restricted (from example disallow read of event title but allow read of the dates)
Whether property can be written to, if user does not have write privileges for the object we can flag all properties as not writable and get extra layer of safety.
On MgdSchema two new parameters to the property are added "read" and "write", both default to "true".
Here we can change a true to false but not vice versa, ie we can add additional restrictions but not remove existing ones, these are added per object instance, for example using the following syntax:
$object->add_acl('property', 'write', false);
Trying to change an existing ACL setting from false to true will throw an exception.
We get signal of an 'event' object being instantiated and we see that this event has 'private' property set to true but the current user is not the creator of the event, so we deny read on event title and some other properties we consider to containt private data.
]]>With Midgard1, especially since the introduction of filesystem-based MidCOM, the deployment model of applications has been quite lacking. In the history of Midgard we have had two deployment models:
Everything served as a repligard XML file, including both content and the application itself (OpenPSA 1.x, Aegir, VMUC, midHoo)
Components installed via PEAR, templates and configuration via FileSync (OpenPSA 2)
Both of these models lack some features we need from a deployment system:
Application: a Midgard-powered web site (combination of components, a page hierarchy, templates and configuration)
Bucket: a repository containing replication XML files of Midgard data, and a manifest file listing necessary components and their versions. Bucket can be either a Amazon S3 bucket or a git repository
Component: a MidCOM component providing its own manifest, and potentially routes, templates, MgdSchema definitions and Python-based helper processes
Alice wants to build her new website with Midgard. She has a Mac laptop, and her ISP has set up a Linux virtual server.
To get started, she downloads the Midgard App Builder and runs it. She instructs the App Builder to set up an empty Midgard application, with only the base MidCOM components installed.
Then she starts her work: she installs the components she needs for the site, converts her XHTML templates to TAL and configures the site structure by using Midgard's on-site administrative tools.
She also sets up a git repository and instructs Midgard to synchronize all changes using that repository as the application bucket.
When the site is ready for launch, she logs on to the Linux server and runs midgard2-setup, providing it with the URL of her application bucket. The Midgard setup tool installs the same set of components that she had on her laptop, imports the site data and sets up a virtual host.
The she tests the site on the actual server and launches it. As the server is monitoring changes to the application bucket, she can later update the website either on the server or on her laptop's local installation. This is useful as she often likes to blog while traveling.
Bob has a customer that has been running a Midgard-powered site for two years now. They contacted him and asked for some layout changes to the site. Since editing a live site is a bad idea, Bob decides to do the changes on a test server.
First he checks if the customer already has their site in an application bucket. But as the site is old, they don't. So he creates an empty bucket to the company version control system and instructs the Midgard installation on the live server to export its data there.
When the whole site is in the bucket together with component information he then logs on to a development server. On the server he already has some other Midgard applications running. In order to not disturb them he decides to set up a new database.
To do this he writes a new Midgard configuration file specifying custom locations for the MgdSchemas, and a new database name. Then he runs midgard2-setup, giving it the name of the configuration file and the URL to the customer's application bucket.
midgard2-setup installs a local copy of the site, exactly like the one running on customer's server. If Bob had been in a hurry, he could have told midgard2-setup to skip installing some slower parts, like for example file attachments.
Now he logs to the copy of the website running on the development server and starts to work.
When the layout changes are done, he then instructs Midgard to export the changes to the customer's application bucket. Then he logs to the live server, and there instructs Midgard to import changes from the bucket. That done, the layout changes are live.
Charlie has built a simple but powerful web-based CRM solution that can be run on the desktop. He wants to start distributing it to his customers, with Mac OS X as the main target platform.
To do this, he creates a new github repository that will server as the application bucket. Then he exports his local installation of the application there, skipping some particular MgdSchema types like "customer" that contain his test data.
After this is done, he fires up Midgard App Builder and tells it to create a new application based on the bucket stored on github. This will result in a clean installation of Charlie's CRM app.
Now he packages that as a .dmg file, publishes it and starts marketing his application. Users who can install and use it just like they would any native Mac OS X app, even though it actually runs Midgard and MidCOM on the background.
When he later releases a new version of the application, he does this by publishing updated component versions and creating an updated manifest file that he then pushes into the application bucket on github.
Users of the CRM application will get automatically notified of the new version as their installations check the github repository periodically for changes. Updating the application is as simple as either running midgard2-setup on the command line, or clicking a button in the application interface itself.
Dexter and Emily are both working in a company providing a social website based on Midgard. They're both developers working on different aspects of the web service.
TODO
In Midgard2, MidCOM components are going to be released via git repositories instead of PEAR packaging. A stable release is made by merging changes from development branches into a stable branch, tagging it and pushing to a repository server. In this sense, midcom_core is also a component as it follows exactly the same model.
Component manifest may list components (repository URL and branch or release tag) the component depends on.
Midgard applications are stored in "Buckets", which can be either git repositories or S3 buckets (other repositories, like WebDAV shares, may also be supported). The bucket contains a manifest file listing the components used by the application, and a collection of Midgard replication XML files grouped by their schema type.
Example bucket layout:
mybucket/
manifest.yml
midgard_person/
e0d837d091c511dba90a8ba58d0b46d346d3.xml
midgard_page
e27c3c8a91c511dbb4287d806e4281ab81ab.xml
e28b76a091c511dbb4287d806e4281ab81ab.xml
...
Tool that handles installation and updating of MidCOM components. When installing a new component, a local git checkout of the component is made (using the given branch or tag), the possible MgdSchemas shipped with the component are installed to Midgard, and the component itself is symlinked to its correct place.
Tool that handles setting up a new Midgard application. It checkouts a given Midgard bucket to the local system, installs the components specified in the bucket manifest (using midgard2-installer routines specified above), and then proceeds to import the Midgard replication XML files in the bucket.
Once components are in place and data has been imported, it then sets up an appropriate virtual host for serving the Midgard application. Once this is done, the Midgard application should be up-and-running and accessible via a browser.
midgard2-setup can either use an existing Midgard configuration file, or set up a new one with sane defaults (based on platform).
Midgard App Builder is a tool based on midgard2 and the Fluid site-specific browser. It makes it possible to easily create and install Midgard applications as self-contained Mac OS X .app packages.
When a Midgard application has been created with App Builder, it will resemble a normal OS X application. Starting it by double-clicking the .app file will start midgard2, lighttpd and d-bus on the background and will open the Midgard application inside a Fluid site-specific browser window. Fluid is based on WebKit and provides many additional javascript APIs that enable applications to behave more like native OS X apps. For instance, they can bounce the dock icon or raise Growl notifications.
When creating a new Midgard application, App Builder should do the normal midgard2-setup tasks (installing the base data from a bucket). After this the fully running Midgard application can then be packaged as a .dmg image that any Mac user will be able to install easily. However, the Midgard bucket subscription on the background will still provide easy update of the base data, for when instance a new release of OpenPSA is made.
Similar tools like Midgard App Builder may be created for other platforms when we have more experiences with real-world usage.
Midgard would be much more cloud-ready if we provided an EC2 image with preinstalled Midgard. When started up, it would connect to a configured bucket and import the application from there.
Then the EC2 node would be completely ready for business. This would make scaling up websites very easy. However, much software needs to be written to make the kind of cloud setup described in an earlier blog post possible:
http://bergie.iki.fi/blog/midgard2-future_in_the_clouds/
Midgard's replication tools require dependencies (objects referenced to via linked fields) to exist in database before objects can be imported. This means that the installer tool must construct a dependency chain and install objects in correct order.
One way to do this is to try importing all objects in sequence. If importing an object fails, move it to the end of the queue and try again later. Repeat until queue is empty. But there may be more optimal solutions to this too.
If a dependency fails to be found completely, we could populate a "skeleton object" into the database. The skeleton object would be completely empty but have the correct GUID, allowing it to be "updated" when the correct object is later being imported.
Packaging and installation of Midgard (and related software like Python, PHP and Lighttpd) is outside the scope of this document.
On Linux systems they can be installed and updated through native package management tools.
Midgard App Builder makes this slightly trickier, as the .app package will contain both Midgard and the actual database. It might be good if Midgard applications generated through App Builder could somehow move their database under user's home directory after initial run. This would make updating Midgard applications on OS X as easy as any native app, as users could just copy a new version of the .app package on top of the existing one and their data would remain intact.
The idea here is the result of much frustration in Midgard's current deployment model. It is partly inspired by discussions in recent Gatherings, and partly by Edd Dumbill's We're all ops people now blog post.
The initial version was written on a rather choppy Austrian Arrows flight from Copenhagen to Vienna, while the plane was working its way around some thunderstorms near Berlin.
]]>With Midgard1, especially since the introduction of filesystem-based MidCOM, the deployment model of applications has been quite lacking. In the history of Midgard we have had two deployment models:
Everything served as a repligard XML file, including both content and the application itself (OpenPSA 1.x, Aegir, VMUC, midHoo)
Components installed via PEAR, templates and configuration via FileSync (OpenPSA 2)
Both of these models lack some features we need from a deployment system:
Application: a Midgard-powered web site (combination of components, a page hierarchy, templates and configuration)
Bucket: a repository containing replication XML files of Midgard data, and a manifest file listing necessary components and their versions. Bucket can be either a Amazon S3 bucket or a git repository
Component: a MidCOM component providing its own manifest, and potentially routes, templates, MgdSchema definitions and Python-based helper processes
Alice wants to build her new website with Midgard. She has a Mac laptop, and her ISP has set up a Linux virtual server.
To get started, she downloads the Midgard App Builder and runs it. She instructs the App Builder to set up an empty Midgard application, with only the base MidCOM components installed.
Then she starts her work: she installs the components she needs for the site, converts her XHTML templates to TAL and configures the site structure by using Midgard's on-site administrative tools.
She also sets up a git repository and instructs Midgard to synchronize all changes using that repository as the application bucket.
When the site is ready for launch, she logs on to the Linux server and runs midgard2-setup, providing it with the URL of her application bucket. The Midgard setup tool installs the same set of components that she had on her laptop, imports the site data and sets up a virtual host.
The she tests the site on the actual server and launches it. As the server is monitoring changes to the application bucket, she can later update the website either on the server or on her laptop's local installation. This is useful as she often likes to blog while traveling.
Bob has a customer that has been running a Midgard-powered site for two years now. They contacted him and asked for some layout changes to the site. Since editing a live site is a bad idea, Bob decides to do the changes on a test server.
First he checks if the customer already has their site in an application bucket. But as the site is old, they don't. So he creates an empty bucket to the company version control system and instructs the Midgard installation on the live server to export its data there.
When the whole site is in the bucket together with component information he then logs on to a development server. On the server he already has some other Midgard applications running. In order to not disturb them he decides to set up a new database.
To do this he writes a new Midgard configuration file specifying custom locations for the MgdSchemas, and a new database name. Then he runs midgard2-setup, giving it the name of the configuration file and the URL to the customer's application bucket.
midgard2-setup installs a local copy of the site, exactly like the one running on customer's server. If Bob had been in a hurry, he could have told midgard2-setup to skip installing some slower parts, like for example file attachments.
Now he logs to the copy of the website running on the development server and starts to work.
When the layout changes are done, he then instructs Midgard to export the changes to the customer's application bucket. Then he logs to the live server, and there instructs Midgard to import changes from the bucket. That done, the layout changes are live.
Charlie has built a simple but powerful web-based CRM solution that can be run on the desktop. He wants to start distributing it to his customers, with Mac OS X as the main target platform.
To do this, he creates a new github repository that will server as the application bucket. Then he exports his local installation of the application there, skipping some particular MgdSchema types like "customer" that contain his test data.
After this is done, he fires up Midgard App Builder and tells it to create a new application based on the bucket stored on github. This will result in a clean installation of Charlie's CRM app.
Now he packages that as a .dmg file, publishes it and starts marketing his application. Users who can install and use it just like they would any native Mac OS X app, even though it actually runs Midgard and MidCOM on the background.
When he later releases a new version of the application, he does this by publishing updated component versions and creating an updated manifest file that he then pushes into the application bucket on github.
Users of the CRM application will get automatically notified of the new version as their installations check the github repository periodically for changes. Updating the application is as simple as either running midgard2-setup on the command line, or clicking a button in the application interface itself.
Dexter and Emily are both working in a company providing a social website based on Midgard. They're both developers working on different aspects of the web service.
TODO
In Midgard2, MidCOM components are going to be released via git repositories instead of PEAR packaging. A stable release is made by merging changes from development branches into a stable branch, tagging it and pushing to a repository server. In this sense, midcom_core is also a component as it follows exactly the same model.
Component manifest may list components (repository URL and branch or release tag) the component depends on.
Midgard applications are stored in "Buckets", which can be either git repositories or S3 buckets (other repositories, like WebDAV shares, may also be supported). The bucket contains a manifest file listing the components used by the application, and a collection of Midgard replication XML files grouped by their schema type.
Example bucket layout:
mybucket/
manifest.yml
midgard_person/
e0d837d091c511dba90a8ba58d0b46d346d3.xml
midgard_page
e27c3c8a91c511dbb4287d806e4281ab81ab.xml
e28b76a091c511dbb4287d806e4281ab81ab.xml
...
Tool that handles installation and updating of MidCOM components. When installing a new component, a local git checkout of the component is made (using the given branch or tag), the possible MgdSchemas shipped with the component are installed to Midgard, and the component itself is symlinked to its correct place.
Tool that handles setting up a new Midgard application. It checkouts a given Midgard bucket to the local system, installs the components specified in the bucket manifest (using midgard2-installer routines specified above), and then proceeds to import the Midgard replication XML files in the bucket.
Once components are in place and data has been imported, it then sets up an appropriate virtual host for serving the Midgard application. Once this is done, the Midgard application should be up-and-running and accessible via a browser.
midgard2-setup can either use an existing Midgard configuration file, or set up a new one with sane defaults (based on platform).
Midgard App Builder is a tool based on midgard2 and the Fluid site-specific browser. It makes it possible to easily create and install Midgard applications as self-contained Mac OS X .app packages.
When a Midgard application has been created with App Builder, it will resemble a normal OS X application. Starting it by double-clicking the .app file will start midgard2, lighttpd and d-bus on the background and will open the Midgard application inside a Fluid site-specific browser window. Fluid is based on WebKit and provides many additional javascript APIs that enable applications to behave more like native OS X apps. For instance, they can bounce the dock icon or raise Growl notifications.
When creating a new Midgard application, App Builder should do the normal midgard2-setup tasks (installing the base data from a bucket). After this the fully running Midgard application can then be packaged as a .dmg image that any Mac user will be able to install easily. However, the Midgard bucket subscription on the background will still provide easy update of the base data, for when instance a new release of OpenPSA is made.
Similar tools like Midgard App Builder may be created for other platforms when we have more experiences with real-world usage.
Midgard would be much more cloud-ready if we provided an EC2 image with preinstalled Midgard. When started up, it would connect to a configured bucket and import the application from there.
Then the EC2 node would be completely ready for business. This would make scaling up websites very easy. However, much software needs to be written to make the kind of cloud setup described in an earlier blog post possible:
http://bergie.iki.fi/blog/midgard2-future_in_the_clouds/
Midgard's replication tools require dependencies (objects referenced to via linked fields) to exist in database before objects can be imported. This means that the installer tool must construct a dependency chain and install objects in correct order.
One way to do this is to try importing all objects in sequence. If importing an object fails, move it to the end of the queue and try again later. Repeat until queue is empty. But there may be more optimal solutions to this too.
If a dependency fails to be found completely, we could populate a "skeleton object" into the database. The skeleton object would be completely empty but have the correct GUID, allowing it to be "updated" when the correct object is later being imported.
Packaging and installation of Midgard (and related software like Python, PHP and Lighttpd) is outside the scope of this document.
On Linux systems they can be installed and updated through native package management tools.
Midgard App Builder makes this slightly trickier, as the .app package will contain both Midgard and the actual database. It might be good if Midgard applications generated through App Builder could somehow move their database under user's home directory after initial run. This would make updating Midgard applications on OS X as easy as any native app, as users could just copy a new version of the .app package on top of the existing one and their data would remain intact.
The idea here is the result of much frustration in Midgard's current deployment model. It is partly inspired by discussions in recent Gatherings, and partly by Edd Dumbill's We're all ops people now blog post.
The initial version was written on a rather choppy Austrian Arrows flight from Copenhagen to Vienna, while the plane was working its way around some thunderstorms near Berlin.
]]>Workspaces is generalized combination of the concepts of SiteGroups and MultiLang, workspaces can be used to replace either or both if so desired on application level and core will be simplified by having only support for the general idea of workspaces.
Workspaces are a tree, they have a local ID (not exposed to developers), a name which must be unique on the current tree level and optionally parent, to set a workspace application calls midgard_connection::set_workspace('/path/to/my/workspace'), workspaces can have record extensions and/or they can be extended via MgdSchema inheritance.
Objects on closer to the root of the tree are inherited towards leaves (read privileges apply), if an object is edited on higher (as in towards leaves) level a copy is made (the copies are full objects on their own, with separate metadata).
Replication is workspace agnostic.
Objects will always have identifiers:
Objects can be instantiated via a midgard_reference object in the following ways:
Objects can be instantiated with their identifier in the following ways:
In both cases core checks if the object has intance in the current workspace, if not it checks the parent etc untill it reaches root, if at any level it encounters an object instance that has read privilege denied it stops checking and throws "access denied" exception, this means that if we have object instances on level 1 and 2 and level 1 has read allowed and level two does not, then on any level above 1 user will not see the object.
Finally a specific instance of object can be instantiated via it's UUID in the following ways:
Cases 2 and 3 basically do 1 transparently. However this way also respects the workspace tree so that if we have UUID '113' in workspace '/foo' and our current workspace is '/baz' we cannot see the object exists, if our workspace was '/foo/bar/baz' we would get the object in workspace '/foo' even if object with same identifier exists in '/foo/bar/baz'.
Replication is workspace agnostic, however it's forbidden to replicate between workspaces in the same database as UUIDs may not clash.
Replication works on object "instances", ie. serializing object instance from workspace '/foo/bar' will not include also the instance from workspace '/foo'.
No workspace info is passed in the serialized data.
For UUIDs that do not yet exist in the database this is simple: importing creates a new copy in the current connection workspace.
If the connection workspace is '/foo/bar/baz' and UUID already exists on '/foo' level then the '/foo' level is updated on import.
If the connection workspace is '/foo/bar/baz' and UUID already exists on '/bar' tree then an error is thrown.
With properly thought out conventions and thinking about the stacking when implementing these, all of the following could be stacked on top of each other so you could have multiple companies with multilingual staging/live sites.
Basically it boils down to what the implementation considers "root" workspace.
For hosting multiple sites of different companies with different needs and data structures using separate databases is probably more efficient or at the very least logically cleaner.
However for application hosting where all customers use mostly the same MgdSchemas workspaces can be used to neatly separate and share data, for example the might be a workspace tree like the following:
Here the huge enterprise and the unrelated company are unaware of each other, data shared across whole enterprise can be created to the first level workspace
For security we should probably have a second argument to set_workspace() which forbids changing the workspace to above/outside the given workspace, so if we say set_workspace('/my_company', true) after that set_workspace('/another_company') should fail but set_workspace('/my_company/fi') should not (provided the workspace exists...), similarly if set_workspace('/my_enterprise/my_company', true) has been called call to set_workspace('/my_enterprise') should fail.
CMS application could handle multi-lingual content by defining that by convention it will look for workspace named for example by locale naming conventions 'fi_FI' for Finnish localized content, core would provide 'fallback language' if the workspaces are organized in the order of '/fallback/preferred_language', this could be a longer path as well for example '/en_US/fi_FI/fi_SV' so we prefer Finnish-swedish but fall back to Finnish if unavailable and finally fall back to english.
User interface locale info could be stored as record extension(s) to the workspace object.
One could have live site use workspace '/My-site' and the staging site use workspace '/My-site/staging' and on publish to live just overwrite the '/My-site' instance of the object with data from the staging instance.
Deletes instead of being done at the same time is from staging (like with the current replication based system) could wait for example re-approval of the parent.
Do we use the update() method to create a copy in the current workspace or do we use a separate method to create the copy ? The con with the update() -approach is that it's "magick", the advantage that component developers do not need to have special handlers.
What happens if we have object with identifier X in '/foo/bar' but not in '/foo' and we want to copy it to '/foo', ie. how to achieve this ?
If we had separate copy method giving the path would be obvious solution. with the update() approach I suppose there is need to change workspace (and if changing workspace above current one is forbidden the separate copy method should fail too).
How should these be handled by core when object is copied from workspace to another ?
Staging/live and/or Multilang applications/helpers handle copying/fallback for attachments.
This is simpler for core and is less "magick" but makes the the required helpers more complicated. Also it can be argued that the record extensions are more part of the object than the attachments.
]]>Workspaces is generalized combination of the concepts of SiteGroups and MultiLang, workspaces can be used to replace either or both if so desired on application level and core will be simplified by having only support for the general idea of workspaces.
Workspaces are a tree, they have a local ID (not exposed to developers), a name which must be unique on the current tree level and optionally parent, to set a workspace application calls midgard_connection::set_workspace('/path/to/my/workspace'), workspaces can have record extensions and/or they can be extended via MgdSchema inheritance.
Objects on closer to the root of the tree are inherited towards leaves (read privileges apply), if an object is edited on higher (as in towards leaves) level a copy is made (the copies are full objects on their own, with separate metadata).
Replication is workspace agnostic.
Objects will always have identifiers:
Objects can be instantiated via a midgard_reference object in the following ways:
Objects can be instantiated with their identifier in the following ways:
In both cases core checks if the object has intance in the current workspace, if not it checks the parent etc untill it reaches root, if at any level it encounters an object instance that has read privilege denied it stops checking and throws "access denied" exception, this means that if we have object instances on level 1 and 2 and level 1 has read allowed and level two does not, then on any level above 1 user will not see the object.
Finally a specific instance of object can be instantiated via it's UUID in the following ways:
Cases 2 and 3 basically do 1 transparently. However this way also respects the workspace tree so that if we have UUID '113' in workspace '/foo' and our current workspace is '/baz' we cannot see the object exists, if our workspace was '/foo/bar/baz' we would get the object in workspace '/foo' even if object with same identifier exists in '/foo/bar/baz'.
Replication is workspace agnostic, however it's forbidden to replicate between workspaces in the same database as UUIDs may not clash.
Replication works on object "instances", ie. serializing object instance from workspace '/foo/bar' will not include also the instance from workspace '/foo'.
No workspace info is passed in the serialized data.
For UUIDs that do not yet exist in the database this is simple: importing creates a new copy in the current connection workspace.
If the connection workspace is '/foo/bar/baz' and UUID already exists on '/foo' level then the '/foo' level is updated on import.
If the connection workspace is '/foo/bar/baz' and UUID already exists on '/bar' tree then an error is thrown.
With properly thought out conventions and thinking about the stacking when implementing these, all of the following could be stacked on top of each other so you could have multiple companies with multilingual staging/live sites.
Basically it boils down to what the implementation considers "root" workspace.
For hosting multiple sites of different companies with different needs and data structures using separate databases is probably more efficient or at the very least logically cleaner.
However for application hosting where all customers use mostly the same MgdSchemas workspaces can be used to neatly separate and share data, for example the might be a workspace tree like the following:
Here the huge enterprise and the unrelated company are unaware of each other, data shared across whole enterprise can be created to the first level workspace
For security we should probably have a second argument to set_workspace() which forbids changing the workspace to above/outside the given workspace, so if we say set_workspace('/my_company', true) after that set_workspace('/another_company') should fail but set_workspace('/my_company/fi') should not (provided the workspace exists...), similarly if set_workspace('/my_enterprise/my_company', true) has been called call to set_workspace('/my_enterprise') should fail.
CMS application could handle multi-lingual content by defining that by convention it will look for workspace named for example by locale naming conventions 'fi_FI' for Finnish localized content, core would provide 'fallback language' if the workspaces are organized in the order of '/fallback/preferred_language', this could be a longer path as well for example '/en_US/fi_FI/fi_SV' so we prefer Finnish-swedish but fall back to Finnish if unavailable and finally fall back to english.
User interface locale info could be stored as record extension(s) to the workspace object.
One could have live site use workspace '/My-site' and the staging site use workspace '/My-site/staging' and on publish to live just overwrite the '/My-site' instance of the object with data from the staging instance.
Deletes instead of being done at the same time is from staging (like with the current replication based system) could wait for example re-approval of the parent.
Do we use the update() method to create a copy in the current workspace or do we use a separate method to create the copy ? The con with the update() -approach is that it's "magick", the advantage that component developers do not need to have special handlers.
What happens if we have object with identifier X in '/foo/bar' but not in '/foo' and we want to copy it to '/foo', ie. how to achieve this ?
If we had separate copy method giving the path would be obvious solution. with the update() approach I suppose there is need to change workspace (and if changing workspace above current one is forbidden the separate copy method should fail too).
How should these be handled by core when object is copied from workspace to another ?
Staging/live and/or Multilang applications/helpers handle copying/fallback for attachments.
This is simpler for core and is less "magick" but makes the the required helpers more complicated. Also it can be argued that the record extensions are more part of the object than the attachments.
]]>Database views implementation should provide flexible way to creating database joins of any type. Views shall be represented by special, abstract midgard_view class. User defined views shall be registered as midgard_view derived classes. Such, shall be available as any other typical classes, and database access shall be encapsulated and completely hidden for them.
Views shall be declared in xml files. View element shall have only one attribute with its name definition. Any other attributes (e.g. with table name) shall not be allowed and considered as explicit error. View shall be defined with reserved word view with reserved name attribute. Implementation shall be responsible for checking name uniqueness. Which means, an explicit error should be raised if view name is already registered as any other class.
<view name="groups_with_coord" />
Table scope, used in FROM SQL part, shall be configured by property's attribute of view. Reserved words table or class shall be defined, and application shall raise error if none of them is found as defined attribute. table attribute shall take precedence, and if not found, class shall be taken into account. A special class attribute defines implicitly table which is defined for given class.
<view name="groups_with_coord" table="grp_table"/>
Implementation shall allow user, to register view properties. These should represent database view fields (columns). Special, reserved word 'property' shall be used to register view property (and thus, field). A reserved, and mandatory word use shall be used as allowed property's element attribute. Such attribute's value shall define class name, and its property which shall be used as database data source. Both shall be separated by semicolon.
<property name="guid" use="midgard_group:guid" />
Optionaly, property element could have description defined as child element. Description shall be defined with reserved word description and with mandatory, reserved value named atribute.
<property name="guid" use="midgard_group:guid">
<decription value="Stores value of a guid which identifies group" />
</property>
User shall be allowed to include metadata properties in a view which is registered. Reserved metadata word shall be used with class and property name to explicitly set metadata column as view's property's storage. Such shall be defined as custom view property, with special dot separator '.' used between metadata keyword and its property name.
<property name="created" use="midgard_group:metadata.created" />
Metadata keyword doesn't describe particular metadata class which is registered as any class' one. User shall be aware of specific metadata class which should be taken into account. Explicit error shall be thrown, if class, which we register view's property for, doesn't have metadata or its metadata doesn't have defined property.
(Note: adding metadata properties is valid since 9.09.2 version)
Views shall be registered during application startup. Midgard unified configuration file shall provide a special key to define path to directory, with view xml files. midgard_view derived classes shall be registered after all user defined MgdSchema classes has been registered. After such step, views shall be created in database.
Reserved, join named element shall define database join. There shall be no limit how many joins could be done per database view. Join type, shall be defined with reserved 'type' attribute. A value for such attribute shall be one of reserved ones:
class named and reserved attribute shall define which, (already defined) class' table is joined in a view. As an alternative, table can be defined explicitly using reserved word table.
<join type="left" class="midgard_member" />
Implementation shall raise error, if given class is not yet registered.
To assure join condition, a special, reserved word condition shall be used as join element's child one. For such, two attributes shall be used: left and right. A value for both shall be class name and its property separated by semicolon.
<join type="left" class="midgard_member">
<condition left="midgard_group:id" right="midgard_member:gid" />
</join>
Optionaly, view definition could have constraints elements to limit records selected from database. This shall be defined with reserved word constraint and with allowed, reserved attributes:
Only value_type shall be optional. This shall define value type, and shall accept the same value types which are accepted in MgdSchema xml files.
<constraint property="midgard_group:owner" operator=">" value="0" value_type="unsigned integer" />
Midgard core shall provide midgard_view abstract class implementation. Such, shall allow programmer:
Every database view represented by midgard_view derived class shall be considered read-only.
midgard_view implementation shall guaranteed sitegroup context. This shall be resolved by implicit constraints or joins usage.
An implicit left join shall be added to every view. Such join shall provide condition, where equality of sitegroup fields is assured.
An implicit constraint shall be added to every view. Such constraint shall limit selected records to these, for which sitegroup field's value is equal to current application sitegroup.
Midgard core shall implement views such way to allow query them with midgard_query_builder or midgard_collector. Optionally midgard_view derived classes shall be usable also with midgard_reflection_property or midgard_replicator. For the latter, only serialization shall be allowed.
]]>Database views implementation should provide flexible way to creating database joins of any type. Views shall be represented by special, abstract midgard_view class. User defined views shall be registered as midgard_view derived classes. Such, shall be available as any other typical classes, and database access shall be encapsulated and completely hidden for them.
Views shall be declared in xml files. View element shall have only one attribute with its name definition. Any other attributes (e.g. with table name) shall not be allowed and considered as explicit error. View shall be defined with reserved word view with reserved name attribute. Implementation shall be responsible for checking name uniqueness. Which means, an explicit error should be raised if view name is already registered as any other class.
<view name="groups_with_coord" />
Table scope, used in FROM SQL part, shall be configured by property's attribute of view. Reserved words table or class shall be defined, and application shall raise error if none of them is found as defined attribute. table attribute shall take precedence, and if not found, class shall be taken into account. A special class attribute defines implicitly table which is defined for given class.
<view name="groups_with_coord" table="grp_table"/>
Implementation shall allow user, to register view properties. These should represent database view fields (columns). Special, reserved word 'property' shall be used to register view property (and thus, field). A reserved, and mandatory word use shall be used as allowed property's element attribute. Such attribute's value shall define class name, and its property which shall be used as database data source. Both shall be separated by semicolon.
<property name="guid" use="midgard_group:guid" />
Optionaly, property element could have description defined as child element. Description shall be defined with reserved word description and with mandatory, reserved value named atribute.
<property name="guid" use="midgard_group:guid">
<decription value="Stores value of a guid which identifies group" />
</property>
User shall be allowed to include metadata properties in a view which is registered. Reserved metadata word shall be used with class and property name to explicitly set metadata column as view's property's storage. Such shall be defined as custom view property, with special dot separator '.' used between metadata keyword and its property name.
<property name="created" use="midgard_group:metadata.created" />
Metadata keyword doesn't describe particular metadata class which is registered as any class' one. User shall be aware of specific metadata class which should be taken into account. Explicit error shall be thrown, if class, which we register view's property for, doesn't have metadata or its metadata doesn't have defined property.
(Note: adding metadata properties is valid since 9.09.2 version)
Views shall be registered during application startup. Midgard unified configuration file shall provide a special key to define path to directory, with view xml files. midgard_view derived classes shall be registered after all user defined MgdSchema classes has been registered. After such step, views shall be created in database.
Reserved, join named element shall define database join. There shall be no limit how many joins could be done per database view. Join type, shall be defined with reserved 'type' attribute. A value for such attribute shall be one of reserved ones:
class named and reserved attribute shall define which, (already defined) class' table is joined in a view. As an alternative, table can be defined explicitly using reserved word table.
<join type="left" class="midgard_member" />
Implementation shall raise error, if given class is not yet registered.
To assure join condition, a special, reserved word condition shall be used as join element's child one. For such, two attributes shall be used: left and right. A value for both shall be class name and its property separated by semicolon.
<join type="left" class="midgard_member">
<condition left="midgard_group:id" right="midgard_member:gid" />
</join>
Optionaly, view definition could have constraints elements to limit records selected from database. This shall be defined with reserved word constraint and with allowed, reserved attributes:
Only value_type shall be optional. This shall define value type, and shall accept the same value types which are accepted in MgdSchema xml files.
<constraint property="midgard_group:owner" operator=">" value="0" value_type="unsigned integer" />
Midgard core shall provide midgard_view abstract class implementation. Such, shall allow programmer:
Every database view represented by midgard_view derived class shall be considered read-only.
midgard_view implementation shall guaranteed sitegroup context. This shall be resolved by implicit constraints or joins usage.
An implicit left join shall be added to every view. Such join shall provide condition, where equality of sitegroup fields is assured.
An implicit constraint shall be added to every view. Such constraint shall limit selected records to these, for which sitegroup field's value is equal to current application sitegroup.
Midgard core shall implement views such way to allow query them with midgard_query_builder or midgard_collector. Optionally midgard_view derived classes shall be usable also with midgard_reflection_property or midgard_replicator. For the latter, only serialization shall be allowed.
]]>Midgard core shall provide possibility to extend existing and already declared MgdSchema classes or copy their data. Two most important cases shall be taken into account in core's implementation:
In this mRFC, Extending or copy MgdSchema class means extending class' on database storage layer.
Extending shall be declared in MgdSchema file with reserved word 'extends'.
<type name="my_note" extends="my_document">
Such inheritance shall add parent's properties to derived type without any need to define them for new type explicitly. Parent's (from which new class extend) properties shall be shared between classes and used as the same storage fields on database layer. In this case, Midgard core shall not make database fields (and especially guids or any other object identifiers) distinction. This should be covered by application which depends on MgdSchema extended classes. Also database integrity (used for replication for example) should be resolved on application level. Metadata shall be shared between classes. In other words, object's property (database field) shall be accessible from parent's class and derived class instance.
TODO: Should we allow multiple child classes?
Copying shall be declared in MgdSchema file with reserved word 'copy'.
<type name="my_note" copy="my_document">
In such case, a deep copy shall be made. Midgard core shall register parent's properties, exactly the same way as they were declared explicitly in MgdSchema file. There shall be no shared properties available and copied properties shall have own fields created in child's class storage.
Properties' name uniqueness shall be implemented explicitly, which means there shall be no implicit or magic implementation which allow to overwrite property being already registered. Any duplicated property name shall be considered an application error.
]]>Midgard core shall provide possibility to extend existing and already declared MgdSchema classes or copy their data. Two most important cases shall be taken into account in core's implementation:
In this mRFC, Extending or copy MgdSchema class means extending class' on database storage layer.
Extending shall be declared in MgdSchema file with reserved word 'extends'.
<type name="my_note" extends="my_document">
Such inheritance shall add parent's properties to derived type without any need to define them for new type explicitly. Parent's (from which new class extend) properties shall be shared between classes and used as the same storage fields on database layer. In this case, Midgard core shall not make database fields (and especially guids or any other object identifiers) distinction. This should be covered by application which depends on MgdSchema extended classes. Also database integrity (used for replication for example) should be resolved on application level. Metadata shall be shared between classes. In other words, object's property (database field) shall be accessible from parent's class and derived class instance.
TODO: Should we allow multiple child classes?
Copying shall be declared in MgdSchema file with reserved word 'copy'.
<type name="my_note" copy="my_document">
In such case, a deep copy shall be made. Midgard core shall register parent's properties, exactly the same way as they were declared explicitly in MgdSchema file. There shall be no shared properties available and copied properties shall have own fields created in child's class storage.
Properties' name uniqueness shall be implemented explicitly, which means there shall be no implicit or magic implementation which allow to overwrite property being already registered. Any duplicated property name shall be considered an application error.
]]>This document describes Midgard rules for creating and maintaining required MidCOM Components Documentation.
Component Documentation must cover the following areas:
Component Documentation must be formatted using Markdown style
General introduction describes component purpose and should be brief but same time should explain the basic operations in a simple non-techie way. User oriented approach should be a main focus in that part of documentation. Introduction shall outline what input component expects, what tasks it does and what output it sends e.g.:
net.nehmer.blog is a newsticker or blog management component. It operates on midgard_article objects stored in midgard_topic container. Component
provides a variety of listing modes including RRS feeds. It also can read RSS from external sources.
Each help_file stands for separate help topic. Upon list of help_files a Table of Contents is created on the fly and attached to Introduction page. Help topics are sorted alphabetically by help_file name therefore it is wise to prepend a filename with a number eg.
01_concept.en.txt
02_advanced_tasks.en.txt
Each help_file must begin with a help topic title that will be taken as subject for Table of Contents and navigation. It is advised that help topic title is formatted as Markdown Header 1 eg.:
My component help topic
---
Every Component that uses styling engine needs to provide a detailed instruction on every style element used. The instruction must contain usage guideliens along with available Component output data and some customization hints.
Article display index
<(index-start)> begins the article index, <(index-item)> displays an individual article and <(index-end)> ends the index. If there are no articles, <(index-empty)> is shown between <(index-start)> and <(index-end)> instead.
Style elements documentation must be midcom.admin.styleeditor compatible.
Starting from version 8.9.3 MgdSchema provides node, a child to node in schema file. Component documentation must include descriptions for all relevant properties so that everyone knows what is the property storage type and purpose. It shall also tell about linkage to other MgdSchema classes.
<property name="relation" type="integer" index="yes">
<description>
Relation options:
- 10: Entry is made at location
- 20: Entry is about location
- 30: Entry is located at location
</description>
</property>
Components that can handle URL routes must provide help on all handlers. There should be a help file containing detailed help. Detailed part is also available from current view, floating toolbar 'Help' menu.
// Handle /list/random/<username|count>/
$this->_request_switch['photostream_list_random_count'] = array
(
'handler' => array ('org_routamc_photostream_handler_list', 'random'),
'fixed_args' => array('random'),
'variable_args' => 1,
);
For the handler above, Component must provide:
URL Methods are standalone scripts that live in exec component folder. URL Methods are launched by running /midcom-exec-{component}/{urlmethod}.php URL.
Documentation on URL Methods is generated in list form:
/midcom-exec-{component}/{urlmethod}.php
urlmethod_{urlmethod}
eg.
/midcom-exec-org.routamc.photostream/import_filesystem_folder.php
urlmethod_import_filesystem_folder
Localization database must contain urlmethod_{urlmethod} identifier and corresponding {urlmethod}.{lang}.txt must exist in documentation folder
]]>This document describes Midgard rules for creating and maintaining required MidCOM Components Documentation.
Component Documentation must cover the following areas:
Component Documentation must be formatted using Markdown style
General introduction describes component purpose and should be brief but same time should explain the basic operations in a simple non-techie way. User oriented approach should be a main focus in that part of documentation. Introduction shall outline what input component expects, what tasks it does and what output it sends e.g.:
net.nehmer.blog is a newsticker or blog management component. It operates on midgard_article objects stored in midgard_topic container. Component
provides a variety of listing modes including RRS feeds. It also can read RSS from external sources.
Each help_file stands for separate help topic. Upon list of help_files a Table of Contents is created on the fly and attached to Introduction page. Help topics are sorted alphabetically by help_file name therefore it is wise to prepend a filename with a number eg.
01_concept.en.txt
02_advanced_tasks.en.txt
Each help_file must begin with a help topic title that will be taken as subject for Table of Contents and navigation. It is advised that help topic title is formatted as Markdown Header 1 eg.:
My component help topic
---
Every Component that uses styling engine needs to provide a detailed instruction on every style element used. The instruction must contain usage guideliens along with available Component output data and some customization hints.
Article display index
<(index-start)> begins the article index, <(index-item)> displays an individual article and <(index-end)> ends the index. If there are no articles, <(index-empty)> is shown between <(index-start)> and <(index-end)> instead.
Style elements documentation must be midcom.admin.styleeditor compatible.
Starting from version 8.9.3 MgdSchema provides node, a child to node in schema file. Component documentation must include descriptions for all relevant properties so that everyone knows what is the property storage type and purpose. It shall also tell about linkage to other MgdSchema classes.
<property name="relation" type="integer" index="yes">
<description>
Relation options:
- 10: Entry is made at location
- 20: Entry is about location
- 30: Entry is located at location
</description>
</property>
Components that can handle URL routes must provide help on all handlers. There should be a help file containing detailed help. Detailed part is also available from current view, floating toolbar 'Help' menu.
// Handle /list/random/<username|count>/
$this->_request_switch['photostream_list_random_count'] = array
(
'handler' => array ('org_routamc_photostream_handler_list', 'random'),
'fixed_args' => array('random'),
'variable_args' => 1,
);
For the handler above, Component must provide:
URL Methods are standalone scripts that live in exec component folder. URL Methods are launched by running /midcom-exec-{component}/{urlmethod}.php URL.
Documentation on URL Methods is generated in list form:
/midcom-exec-{component}/{urlmethod}.php
urlmethod_{urlmethod}
eg.
/midcom-exec-org.routamc.photostream/import_filesystem_folder.php
urlmethod_import_filesystem_folder
Localization database must contain urlmethod_{urlmethod} identifier and corresponding {urlmethod}.{lang}.txt must exist in documentation folder
]]>As D-Bus allows to create self, freely defined paths for objects, it might be confusing to send messages to other objects without any knowledge about their particular paths. This document defines D-Bus messages and objects' paths, at which instances has been created.
'midgard_article' class name is used in every example below.
Services objects can be created by any application as long as they conform to this mRFC.
Clients objects can be created by any application as long as they conform to this mRFC. For asynchronous massages, Midgard API shall provide automatic messages for main I/0 operations.
To limit possible collisions and provide clear and safe services' and clients' identifiers, guid of the current sitegroup is appended to main service's path. Midgard API implementation should not require to append such guid by application, and path with configuration name should be created automatically.
For example, if application is running in some particular sitegroup context, the base path for services and clients shall look like this:
/org/midgardproject/1de0f4967e7ac080f4911debfa73b76220858805880
Midgard application should not be aware of sitegroup guid as both service and client will be created for the same base path. Midgard API for D-Bus should require only module or class names.
Sitegroup (domain) 0 is a special exception and SG0 string is used instead of sitegroup's guid.
Services' paths should be created with classes' names, and if possible with module's or feature's names to separate services objects. For example all user defined MgdSchema classes should be created for 'MgdSchema' path. Class method should be appended to path to finally identify service's name.
This document must define clear path's method name, if class' real method name can not be appended to path or any other name suits better.
Services for MgdSchema should be created at path with following tokens:
Every mgdschema class' service or client shall be initialized with the same convention.
'create' shall be appended to object's path.
/mgdschema/midgard_article/create
On success, Midgard API shall send asynchronous message to object at path.
'update' shall be appended to object's path
/mgdschema/midgard_article/update
On success, Midgard API shall send asynchronous message to object at path.
'delete' shall be appended to object's path
/mgdschema/midgard_article/delete
On success, Midgard API shall send asynchronous message to object at path.
'purge' shall be appended to object's path
/mgdschema/midgard_article/purge
On success, Midgard API shall send asynchronous message to object at path.
'get' shall be appended to object's path for one of these methods:
get_parent
/mgdschema/midgard_article/get
Midgard API implementation will send asynchronous messages in above cases, only if object is successfully retrieved from database.
Services for replication should be created at path with following tokens:
/replication/midgard_article/import
/replication/midgard_article/export
Messages sent to Midgard D-Bus services should contain:
As D-Bus allows to create self, freely defined paths for objects, it might be confusing to send messages to other objects without any knowledge about their particular paths. This document defines D-Bus messages and objects' paths, at which instances has been created.
'midgard_article' class name is used in every example below.
Services objects can be created by any application as long as they conform to this mRFC.
Clients objects can be created by any application as long as they conform to this mRFC. For asynchronous massages, Midgard API shall provide automatic messages for main I/0 operations.
To limit possible collisions and provide clear and safe services' and clients' identifiers, guid of the current sitegroup is appended to main service's path. Midgard API implementation should not require to append such guid by application, and path with configuration name should be created automatically.
For example, if application is running in some particular sitegroup context, the base path for services and clients shall look like this:
/org/midgardproject/1de0f4967e7ac080f4911debfa73b76220858805880
Midgard application should not be aware of sitegroup guid as both service and client will be created for the same base path. Midgard API for D-Bus should require only module or class names.
Sitegroup (domain) 0 is a special exception and SG0 string is used instead of sitegroup's guid.
Services' paths should be created with classes' names, and if possible with module's or feature's names to separate services objects. For example all user defined MgdSchema classes should be created for 'MgdSchema' path. Class method should be appended to path to finally identify service's name.
This document must define clear path's method name, if class' real method name can not be appended to path or any other name suits better.
Services for MgdSchema should be created at path with following tokens:
Every mgdschema class' service or client shall be initialized with the same convention.
'create' shall be appended to object's path.
/mgdschema/midgard_article/create
On success, Midgard API shall send asynchronous message to object at path.
'update' shall be appended to object's path
/mgdschema/midgard_article/update
On success, Midgard API shall send asynchronous message to object at path.
'delete' shall be appended to object's path
/mgdschema/midgard_article/delete
On success, Midgard API shall send asynchronous message to object at path.
'purge' shall be appended to object's path
/mgdschema/midgard_article/purge
On success, Midgard API shall send asynchronous message to object at path.
'get' shall be appended to object's path for one of these methods:
get_parent
/mgdschema/midgard_article/get
Midgard API implementation will send asynchronous messages in above cases, only if object is successfully retrieved from database.
Services for replication should be created at path with following tokens:
/replication/midgard_article/import
/replication/midgard_article/export
Messages sent to Midgard D-Bus services should contain:
2007-12-05 Created by Piotr Pokora
This document describes Midgard objects' name uniqueness, scope of uniqueness and configuration possibilities. It is also part of Midgard Tree functionality.
Whenever name word is used in this document it refers to object property registered with MgdSchema and configured to be unique in particular scope.
Unique name should be a property of any type, however it's recommended to use string type with unique names. Name should be configured in Midgard Schema xml file and there should be only one unique name defined per class ( type ). Name should be configured only for classes which may be represented in Midgard Tree. For any other class, unique name should be ignored.
Reserved configuration attribute which defines unique name itself, doesn't qualifies a class ( for which name belongs to) as Midgard Tree enabled.
Name uniqueness should not be supported by underlying database storage provider, unless such functionality would not break sitegroup or Midgard Tree specifications.
Unique name should be configured only with reserved unique attribute(s) in xml file. A value for the given attribute should define 3 different scopes of uniquness:
It guarantees a name is unique in scope of parent type object's identifier.
It guarantees a name is unique in scope of the same type object's identifier.
It guarantees a name is unique in scope of the same and parent type object's identifier.
<property name="title" unique="both">
<property name="name" unique="parent">
OR Unique name should be configured only with reserved unique and scope attribute(s) in xml file.
<property name="title" unique="yes" unique_scope="both">
<property name="name" unique="yes" unique_scope="self">
Describe parent name uniqueness enforce when value is set to 'both'.
2007-12-05 Created by Piotr Pokora
This document describes Midgard objects' name uniqueness, scope of uniqueness and configuration possibilities. It is also part of Midgard Tree functionality.
Whenever name word is used in this document it refers to object property registered with MgdSchema and configured to be unique in particular scope.
Unique name should be a property of any type, however it's recommended to use string type with unique names. Name should be configured in Midgard Schema xml file and there should be only one unique name defined per class ( type ). Name should be configured only for classes which may be represented in Midgard Tree. For any other class, unique name should be ignored.
Reserved configuration attribute which defines unique name itself, doesn't qualifies a class ( for which name belongs to) as Midgard Tree enabled.
Name uniqueness should not be supported by underlying database storage provider, unless such functionality would not break sitegroup or Midgard Tree specifications.
Unique name should be configured only with reserved unique attribute(s) in xml file. A value for the given attribute should define 3 different scopes of uniquness:
It guarantees a name is unique in scope of parent type object's identifier.
It guarantees a name is unique in scope of the same type object's identifier.
It guarantees a name is unique in scope of the same and parent type object's identifier.
<property name="title" unique="both">
<property name="name" unique="parent">
OR Unique name should be configured only with reserved unique and scope attribute(s) in xml file.
<property name="title" unique="yes" unique_scope="both">
<property name="name" unique="yes" unique_scope="self">
Describe parent name uniqueness enforce when value is set to 'both'.
It is possible to extend the Asgard functionality by adding other modules to the system.
Components wishing to operate as Asgard plugins must have a class that registers plugin handler classes to the request switch. This is defined in component manifest via:
'customdata' => array
(
'asgard_plugin' => array
(
'class' => 'net_example_adminmodule_request',
'src' => 'file:/net/example/adminmodule/request.php',
'name' => 'Example admin',
'config' => null,
),
),
The actual class must have a method get_plugin_handlers() that returns a request switch array and initiates the Asgard library:
function get_plugin_handlers()
{
$_MIDCOM->load_library('midgard.admin.asgard');
$_MIDCOM->auth->require_valid_user();
return Array
(
'index' => Array
(
'handler' => array('net_example_adminmodule_handler_main', 'main'),
),
);
}
Note: There must be handler for a view without arguments as this will be the screen users enter when choosing the module from Asgard navigation.
The handler class must populate some data used by Asgard.
During the handle phase the plugin must register itself to Asgard:
$my_title = $_MIDCOM->i18n->get_string('example admin plugin', 'net.example.adminplugin');
midgard_admin_asgard_plugin::prepare_plugin($my_title, &$data);
During show phase the element must show Asgard's elements around its own content:
midcom_show_style('midgard_admin_asgard_header');
midcom_show_style('midgard_admin_asgard_middle');
// Show plugin's own output
midcom_show_style('midgard_admin_asgard_footer');
It is possible to extend the Asgard functionality by adding other modules to the system.
Components wishing to operate as Asgard plugins must have a class that registers plugin handler classes to the request switch. This is defined in component manifest via:
'customdata' => array
(
'asgard_plugin' => array
(
'class' => 'net_example_adminmodule_request',
'src' => 'file:/net/example/adminmodule/request.php',
'name' => 'Example admin',
'config' => null,
),
),
The actual class must have a method get_plugin_handlers() that returns a request switch array and initiates the Asgard library:
function get_plugin_handlers()
{
$_MIDCOM->load_library('midgard.admin.asgard');
$_MIDCOM->auth->require_valid_user();
return Array
(
'index' => Array
(
'handler' => array('net_example_adminmodule_handler_main', 'main'),
),
);
}
Note: There must be handler for a view without arguments as this will be the screen users enter when choosing the module from Asgard navigation.
The handler class must populate some data used by Asgard.
During the handle phase the plugin must register itself to Asgard:
$my_title = $_MIDCOM->i18n->get_string('example admin plugin', 'net.example.adminplugin');
midgard_admin_asgard_plugin::prepare_plugin($my_title, &$data);
During show phase the element must show Asgard's elements around its own content:
midcom_show_style('midgard_admin_asgard_header');
midcom_show_style('midgard_admin_asgard_middle');
// Show plugin's own output
midcom_show_style('midgard_admin_asgard_footer');
Currently MidCOM loads a huge number of libraries and services for each request, requiring large amounts of file accesses and PHP parsing. Most of the services loaded are however not really used in a typical request, and so can be seen as being loaded "just in case".
The belief is that this makes the MidCOM environment heavier and more difficult to understand. This proposal outlines how MidCOM could migrate from the current model to a model where services would be loaded when they are used for the first time.
A new class midcom_helper_serviceloader needs to be defined to act as the access point to the MidCOM service system. It will be instantiated as $_MIDCOM->serviceloader at MidCOM start-up.
The service loader should provide two public methods:
boolean can_load($service)
Method for checking if a particular service is available on the system. This would enable components to deal with missing requirements or other situations where some functionality is not available more cleanly.
object load($service)
Method for instantiating an implementation of a particular service.
An additional benefit of the service loader model is that MidCOM's services can start following the Inversion of Control design pattern.
This means that services are defined by their interface classes, but actual implementation of a service can be chosen by site administrator. This will enable us to easily try and swap between various ways of doing things.
Example: URL name generation can be moved to a midcom_core_service_urlgenerator service. Since the default URL name generator shipping with MidCOM does not deal particularly well with the Georgian language, a web developer deploying Midgard in Georgia can write their own service implementation and just swap that to be used in site configuration.
$GLOBALS['midcom_config_local']['service_midcom_core_service_urlgenerator'] = 'ge_convert_urlgenerator';
The two-tiered approach of having interface and implementation separately means that two files have to be loaded for each service. This causes a slight performance hit when compared to current approach of one file per service.
However, it is believed that the benefit of loading services only when they're needed and being able to change service implementations on configuration level easily offsets this.
The actual service loader and an URL generator service have already been implemented as an experiment in MidCOM trunk. If this proposal is accepted, all services and "helper libraries" in MidCOM need to be refactored to the new model and components switched to use them.
As this is a big task, the proposal is to switch services to the new model one-by-one, starting from libraries that are used in only few places.
As of 2007-07-19, MidCOM loads 57 files in the midcom.php, with many of them potentially loading sub-dependencies. Most of these could be moved to on-demand handling or removed by the separately proposed DBA rewrite.
Currently MidCOM loads a huge number of libraries and services for each request, requiring large amounts of file accesses and PHP parsing. Most of the services loaded are however not really used in a typical request, and so can be seen as being loaded "just in case".
The belief is that this makes the MidCOM environment heavier and more difficult to understand. This proposal outlines how MidCOM could migrate from the current model to a model where services would be loaded when they are used for the first time.
A new class midcom_helper_serviceloader needs to be defined to act as the access point to the MidCOM service system. It will be instantiated as $_MIDCOM->serviceloader at MidCOM start-up.
The service loader should provide two public methods:
boolean can_load($service)
Method for checking if a particular service is available on the system. This would enable components to deal with missing requirements or other situations where some functionality is not available more cleanly.
object load($service)
Method for instantiating an implementation of a particular service.
An additional benefit of the service loader model is that MidCOM's services can start following the Inversion of Control design pattern.
This means that services are defined by their interface classes, but actual implementation of a service can be chosen by site administrator. This will enable us to easily try and swap between various ways of doing things.
Example: URL name generation can be moved to a midcom_core_service_urlgenerator service. Since the default URL name generator shipping with MidCOM does not deal particularly well with the Georgian language, a web developer deploying Midgard in Georgia can write their own service implementation and just swap that to be used in site configuration.
$GLOBALS['midcom_config_local']['service_midcom_core_service_urlgenerator'] = 'ge_convert_urlgenerator';
The two-tiered approach of having interface and implementation separately means that two files have to be loaded for each service. This causes a slight performance hit when compared to current approach of one file per service.
However, it is believed that the benefit of loading services only when they're needed and being able to change service implementations on configuration level easily offsets this.
The actual service loader and an URL generator service have already been implemented as an experiment in MidCOM trunk. If this proposal is accepted, all services and "helper libraries" in MidCOM need to be refactored to the new model and components switched to use them.
As this is a big task, the proposal is to switch services to the new model one-by-one, starting from libraries that are used in only few places.
As of 2007-07-19, MidCOM loads 57 files in the midcom.php, with many of them potentially loading sub-dependencies. Most of these could be moved to on-demand handling or removed by the separately proposed DBA rewrite.
2007-07-05 Created by Piotr Pokora
Generic membership describes the basic way to group any MidgardSchema ( MgdSchema ) type ( class ) as an member of any other type registered in Midgard application. This mRFC also describes known Midgard functionality to group midgard_persons objects as a members of midgard_group one, and capabilities of membership introspection.
The type which groups other types. One type ( class ) can not group objects of the same type.
The type which belongs to owner group. One type ( class ) can not belongs to group of the same type.
The type which holds required information ( like owner or member identifier ) for members and owners.
Membership information shall be defined in xml schema file as a type properties and attributes and particular introspection capabilities shall be implemented on API level. Membership shall be always defined for member type and never for owner type.
Minimal information should contain:
<type name="midgard_person" table="person" parentfield="creator">
<membership name="midgard_member" owner="midgard_group" >
Membership implementation should allow to retrieve following information:
In general, members should be created with particular membership type implementation, to avoid direct and random database access. Membership type should always create or delete member for given member and owner type.
]]>2007-07-05 Created by Piotr Pokora
Generic membership describes the basic way to group any MidgardSchema ( MgdSchema ) type ( class ) as an member of any other type registered in Midgard application. This mRFC also describes known Midgard functionality to group midgard_persons objects as a members of midgard_group one, and capabilities of membership introspection.
The type which groups other types. One type ( class ) can not group objects of the same type.
The type which belongs to owner group. One type ( class ) can not belongs to group of the same type.
The type which holds required information ( like owner or member identifier ) for members and owners.
Membership information shall be defined in xml schema file as a type properties and attributes and particular introspection capabilities shall be implemented on API level. Membership shall be always defined for member type and never for owner type.
Minimal information should contain:
<type name="midgard_person" table="person" parentfield="creator">
<membership name="midgard_member" owner="midgard_group" >
Membership implementation should allow to retrieve following information:
In general, members should be created with particular membership type implementation, to avoid direct and random database access. Membership type should always create or delete member for given member and owner type.
]]>This mRFC describes xml file format which should be used for objects' replication, and for data exchange between Midgard and other systems. It is impossible to define xml file format for every type registered in Midgard type system , however this mRFC focuses on minimal xml schema (2) file which should describe serialized Midgard object. Xml file with serialized Midgard Object should contain Midgard namespace, Midgard Object type name, optionally its guid and lang which identify object and common metadata.
Midgard namespace should be defined as: __http://www.midgard-project.org/midgard_object/1.8__
( Optionally namespace could define Midgard version for which given xml file can be used )
Midgard type should be defined as complexType identified by name of string type. Optionally 'guid' and 'lang' atributes could be used to identify Midgard Object. Guid attribute if used should be at least 28 and at most 80 characters long (1). Lang (3) attribute should be 2 letter language code name and define available languages enumeration.
For better understanding and connectivity , midgard-core should have implemented functionality to create xml schema file for any type registered in Midgard Schema. Additionally this implementation should create languages enumeration for all available languages (3) in Midgard environment. Every Midgard object's property should be defined as element type with optional attributes.
If serialized object is multilingual Midgard type and more than one language is used for such object , all language content objects should be serialized and stored in one xml file. In such case deserialization implementation should return array of multilingual objects, and lang attribute's value should be mandatory set in xml file.
Optionally , object's paramaters ( registered as standalone midgard_parameter type ) and object's attachments ( registered as standalone midgard_attachment type ) could be serialized and stored in object's xml file.
<?xml version="1.0"?>
<?xml version="1.0" encoding="UTF-8"?>
<midgard_object xmlns="http://www.midgard-project.org/midgard_object/1.8">
<midgard_article guid="26b621b938c4aea0e021d4baae69d6ab" lang="en">
<metadata>
<creator></creator>
<created>1999-05-07 18:04:04</created>
<revisor></revisor>
<revised>2006-09-10 09:28:08</revised>
<revision>1</revision>
<locker></locker>
<locked>1999-08-17 16:01:32</locked>
<approver></approver>
<approved></approved>
<authors>f6b665f1984503790ed91f39b11b5392</authors>
<owner></owner>
<schedulestart></schedulestart>
<scheduleend></scheduleend>
<hidden>0</hidden>
<navnoentry>0</navnoentry>
<size>481</size>
<published>1999-05-07 18:04:04</published>
<score>0</score>
<imported></imported>
<exported>2006-09-14 12:37:40+0000</exported>
<deleted>0</deleted>
</metadata>
<action></action>
<id>2</id>
<created>1999-05-07 18:04:04</created>
<revisor>f6b665f1984503790ed91f39b11b5392</revisor>
<score>0</score>
<title>Host Administration</title>
<abstract>Midgard host and directory management
</abstract>
<locker>f6b665f1984503790ed91f39b11b5392</locker>
<caldays>0</caldays>
<approved></approved>
<calstart>0000-00-00</calstart>
<content><p>
Use the host administration tool to create and maintain the structure and functionality of your web sites. Use the layout adminstration tool for controlling the appearance of the site and the content administration tool for the dynamic content of the site. </content>
<icon>0</icon>
<type>0</type>
<creator>f6b665f1984503790ed91f39b11b53922</creator>
<revised></revised>
<approver>0</approver>
<extra1></extra1>
<name></name>
<author>f6b665f1984503790ed91f39b11b5392</author>
<url></url>
<contentauthor>1</contentauthor>
<contentcreated>1999-05-07 18:04:04</contentcreated>
<print>0</print>
<extra2></extra2>
<view>0</view>
<extra3></extra3>
<revision>0</revision>
<sid>2</sid>
<locked>1999-08-17 16:01:32</locked>
<up>d0099a92c1018461bc6e33cd0a1c435b</up>
</midgard_article>
</midgard_object>
This mRFC describes xml file format which should be used for objects' replication, and for data exchange between Midgard and other systems. It is impossible to define xml file format for every type registered in Midgard type system , however this mRFC focuses on minimal xml schema (2) file which should describe serialized Midgard object. Xml file with serialized Midgard Object should contain Midgard namespace, Midgard Object type name, optionally its guid and lang which identify object and common metadata.
Midgard namespace should be defined as: __http://www.midgard-project.org/midgard_object/1.8__
( Optionally namespace could define Midgard version for which given xml file can be used )
Midgard type should be defined as complexType identified by name of string type. Optionally 'guid' and 'lang' atributes could be used to identify Midgard Object. Guid attribute if used should be at least 28 and at most 80 characters long (1). Lang (3) attribute should be 2 letter language code name and define available languages enumeration.
For better understanding and connectivity , midgard-core should have implemented functionality to create xml schema file for any type registered in Midgard Schema. Additionally this implementation should create languages enumeration for all available languages (3) in Midgard environment. Every Midgard object's property should be defined as element type with optional attributes.
If serialized object is multilingual Midgard type and more than one language is used for such object , all language content objects should be serialized and stored in one xml file. In such case deserialization implementation should return array of multilingual objects, and lang attribute's value should be mandatory set in xml file.
Optionally , object's paramaters ( registered as standalone midgard_parameter type ) and object's attachments ( registered as standalone midgard_attachment type ) could be serialized and stored in object's xml file.
<?xml version="1.0"?>
<?xml version="1.0" encoding="UTF-8"?>
<midgard_object xmlns="http://www.midgard-project.org/midgard_object/1.8">
<midgard_article guid="26b621b938c4aea0e021d4baae69d6ab" lang="en">
<metadata>
<creator></creator>
<created>1999-05-07 18:04:04</created>
<revisor></revisor>
<revised>2006-09-10 09:28:08</revised>
<revision>1</revision>
<locker></locker>
<locked>1999-08-17 16:01:32</locked>
<approver></approver>
<approved></approved>
<authors>f6b665f1984503790ed91f39b11b5392</authors>
<owner></owner>
<schedulestart></schedulestart>
<scheduleend></scheduleend>
<hidden>0</hidden>
<navnoentry>0</navnoentry>
<size>481</size>
<published>1999-05-07 18:04:04</published>
<score>0</score>
<imported></imported>
<exported>2006-09-14 12:37:40+0000</exported>
<deleted>0</deleted>
</metadata>
<action></action>
<id>2</id>
<created>1999-05-07 18:04:04</created>
<revisor>f6b665f1984503790ed91f39b11b5392</revisor>
<score>0</score>
<title>Host Administration</title>
<abstract>Midgard host and directory management
</abstract>
<locker>f6b665f1984503790ed91f39b11b5392</locker>
<caldays>0</caldays>
<approved></approved>
<calstart>0000-00-00</calstart>
<content><p>
Use the host administration tool to create and maintain the structure and functionality of your web sites. Use the layout adminstration tool for controlling the appearance of the site and the content administration tool for the dynamic content of the site. </content>
<icon>0</icon>
<type>0</type>
<creator>f6b665f1984503790ed91f39b11b53922</creator>
<revised></revised>
<approver>0</approver>
<extra1></extra1>
<name></name>
<author>f6b665f1984503790ed91f39b11b5392</author>
<url></url>
<contentauthor>1</contentauthor>
<contentcreated>1999-05-07 18:04:04</contentcreated>
<print>0</print>
<extra2></extra2>
<view>0</view>
<extra3></extra3>
<revision>0</revision>
<sid>2</sid>
<locked>1999-08-17 16:01:32</locked>
<up>d0099a92c1018461bc6e33cd0a1c435b</up>
</midgard_article>
</midgard_object>
Midgard's base of power is Europe where there are many languages and many organizations operating across language barriers. This makes it very important to support easy management of multilingual sites.
Traditionally there have been two different types of multilingual sites:
This proposal focuses on the first of these two. Multiple language versions of a site are better managed using separate content trees for each site than using MultiLang features.
Right from the beginnings of the project, Midgard has had a strong international focus. Multibyte character set support landed in Midgard already in October 1999 in order to support managing sites in languages like Russian and Chinese. UTF-8 was made the default character set for Midgard in the 1.6 series.
However, in addition to character sets, also translation of actual content was needed. MultiLingual Midgard was presented as a patch by David Schmitter from DataFlow in Switzerland in May 2003, and entered Midgard proper for the 1.5.0 release.
This made the Midgard content formats support different translations of content, but besides DataFlow's proprietary WebInOne publishing tool, no Midgard authoring interface has actually added MultiLang support.
The reason for not supporting MultiLang has mostly been the difficulty of integrating good translation workflow, and unclear PHP-level programming APIs for it. The APIs have now however matured enough with integration to MgdSchema and Query Builder tools so that supporting MultiLingual content cleanly is finally possible.
Here is a description on how MultiLang should be implemented into the Midgard Component Framework. The aim here is to make creating and maintaining multilingual sites easy without having to impose changes into MidCOM components.
Webite's language selections (languages available for the site and the default language) are stored in the MidCOM configuration array.
The language selections can be made using Site Wizard for better convenience
Languages would be selected using the $argv[0] prefix of the site, i.e. Finnish version of "About us" would reside in /fi/about. This can be implemented in a centralized fashion by modifying MidCOM's URL parser and making it call the corresponding MidCOM i18n service set_language method for changing language.
In addition to the per-language URLs, there would be the "language neutral" URL for each object. This would redirect user to appropriate URL based on their browser language selection, i.e. /about would redirect me to /de/about for German language users.
Language versions of site should obviously show only the contents translated to their languages in navigation and content listings (like news lists for instance). Since MidCOM has a centralized wrapper for Query Builder, the DBA can easily add the language constraint into all DB queries transparently. This means components don't have to do any language-based processing to show only Norwegian news items.
Similarly, when an object is instantiated (get_by_guid etc), DBA should check if the $object->lang is the current language and if not fail the instantiation.
MidCOM should contain a specific "translation to this language not found" error message that could be triggered if requested object is not available in current language. This error message could contain links to the existing language versions.
Since editing MidCOM content now happens on-site and there are language prefixes available, switching editing views between languages is just a matter of changing the site prefix.
However, there should be some facilities for showing to translators what has been changes in the "default language" contents of an object so they know what to translate. This relates to RCS version control services in MidCOM.
MidCOM's i18n service has already been modified so that user interface language can be different from actual content language. Many multilingual organizations prefer to run their software in a common language to ease documentation and training.
The information on different language versions of an object must be available so that MidCOM's metadata system can populate links to the different translations.
Here is a list of requirements from the MultiLang implementation in Midgard Core that are needed for this mRFC:
midgard_language objects must be available for regular Query Builder operations, and must contain a locale field with the appropriate UNIX system localemgd_set_lang(X) has been called, all object update/create/delete operations must apply to the content of that particular language$object->get_languages() must return list of languages the object is available forset_lang has been called for Query Builder, it must only return objects that are in that particular language, no lang0 objectsMidgard's base of power is Europe where there are many languages and many organizations operating across language barriers. This makes it very important to support easy management of multilingual sites.
Traditionally there have been two different types of multilingual sites:
This proposal focuses on the first of these two. Multiple language versions of a site are better managed using separate content trees for each site than using MultiLang features.
Right from the beginnings of the project, Midgard has had a strong international focus. Multibyte character set support landed in Midgard already in October 1999 in order to support managing sites in languages like Russian and Chinese. UTF-8 was made the default character set for Midgard in the 1.6 series.
However, in addition to character sets, also translation of actual content was needed. MultiLingual Midgard was presented as a patch by David Schmitter from DataFlow in Switzerland in May 2003, and entered Midgard proper for the 1.5.0 release.
This made the Midgard content formats support different translations of content, but besides DataFlow's proprietary WebInOne publishing tool, no Midgard authoring interface has actually added MultiLang support.
The reason for not supporting MultiLang has mostly been the difficulty of integrating good translation workflow, and unclear PHP-level programming APIs for it. The APIs have now however matured enough with integration to MgdSchema and Query Builder tools so that supporting MultiLingual content cleanly is finally possible.
Here is a description on how MultiLang should be implemented into the Midgard Component Framework. The aim here is to make creating and maintaining multilingual sites easy without having to impose changes into MidCOM components.
Webite's language selections (languages available for the site and the default language) are stored in the MidCOM configuration array.
The language selections can be made using Site Wizard for better convenience
Languages would be selected using the $argv[0] prefix of the site, i.e. Finnish version of "About us" would reside in /fi/about. This can be implemented in a centralized fashion by modifying MidCOM's URL parser and making it call the corresponding MidCOM i18n service set_language method for changing language.
In addition to the per-language URLs, there would be the "language neutral" URL for each object. This would redirect user to appropriate URL based on their browser language selection, i.e. /about would redirect me to /de/about for German language users.
Language versions of site should obviously show only the contents translated to their languages in navigation and content listings (like news lists for instance). Since MidCOM has a centralized wrapper for Query Builder, the DBA can easily add the language constraint into all DB queries transparently. This means components don't have to do any language-based processing to show only Norwegian news items.
Similarly, when an object is instantiated (get_by_guid etc), DBA should check if the $object->lang is the current language and if not fail the instantiation.
MidCOM should contain a specific "translation to this language not found" error message that could be triggered if requested object is not available in current language. This error message could contain links to the existing language versions.
Since editing MidCOM content now happens on-site and there are language prefixes available, switching editing views between languages is just a matter of changing the site prefix.
However, there should be some facilities for showing to translators what has been changes in the "default language" contents of an object so they know what to translate. This relates to RCS version control services in MidCOM.
MidCOM's i18n service has already been modified so that user interface language can be different from actual content language. Many multilingual organizations prefer to run their software in a common language to ease documentation and training.
The information on different language versions of an object must be available so that MidCOM's metadata system can populate links to the different translations.
Here is a list of requirements from the MultiLang implementation in Midgard Core that are needed for this mRFC:
midgard_language objects must be available for regular Query Builder operations, and must contain a locale field with the appropriate UNIX system localemgd_set_lang(X) has been called, all object update/create/delete operations must apply to the content of that particular language$object->get_languages() must return list of languages the object is available forset_lang has been called for Query Builder, it must only return objects that are in that particular language, no lang0 objectsCache approaches:
MidgardApacheCache should extend base MidgardCache class, and should be optimized for any midgard extension which is used in Apache module environment. MidgardCache base class should be assigned as private member of MidgardConnection and thus connection handler object could be responsible for managing internal's core objects references counts.
MidgardApacheCache when instanciated should return UUID which should be available during the whole application runtime. For Apache module, it means - as long as httpd server is running.
There shouldn't be any method or function in implementation which should allow list all cache entries ( MidgardApacheCache instances ) available.
Retrieving MidgardApacheCache instance should be possible only with UUID representing particular object instance and application which use cache object should be responsible to store this UUID.
<?php
$application_cache_guid = "guid_string";
$cache = new midgard_apache_cache($application_cache_guid);
if(!$cache->exists())
$application_cache_guid = $cache->create();
else
$cache->get();
?>
The main purpose of MidgardApacheCache should be possibility to store objects , however other types should be supported as well. Objects should be identified by guids in the same way as are identified and retrieved in typical Midgard applications. Other types like arrays should be registered with unique names.
<?php
$object = $cache->get_type_by_guid("object_guid_string");
$object->register("my_constant_array", $array_type);
$array = $object->get_type_by_name("my_constant_array");
?>
MidgardApacheCache should implement such features:
Cache approaches:
MidgardApacheCache should extend base MidgardCache class, and should be optimized for any midgard extension which is used in Apache module environment. MidgardCache base class should be assigned as private member of MidgardConnection and thus connection handler object could be responsible for managing internal's core objects references counts.
MidgardApacheCache when instanciated should return UUID which should be available during the whole application runtime. For Apache module, it means - as long as httpd server is running.
There shouldn't be any method or function in implementation which should allow list all cache entries ( MidgardApacheCache instances ) available.
Retrieving MidgardApacheCache instance should be possible only with UUID representing particular object instance and application which use cache object should be responsible to store this UUID.
<?php
$application_cache_guid = "guid_string";
$cache = new midgard_apache_cache($application_cache_guid);
if(!$cache->exists())
$application_cache_guid = $cache->create();
else
$cache->get();
?>
The main purpose of MidgardApacheCache should be possibility to store objects , however other types should be supported as well. Objects should be identified by guids in the same way as are identified and retrieved in typical Midgard applications. Other types like arrays should be registered with unique names.
<?php
$object = $cache->get_type_by_guid("object_guid_string");
$object->register("my_constant_array", $array_type);
$array = $object->get_type_by_name("my_constant_array");
?>
MidgardApacheCache should implement such features:
This mRFC is a proposal for repligard functionality replacement , supported by midgard-core and by any language for which Midgard language bindings exist. As repligard supported Midgard database replication without any possibility to change its behaviour a lot , this mRFC focuses on object's records replication , object exporting and object importing to or from any database storage. This mRFC also describes possibility to undelete and purge object's record(s).
Examples in this mRFC use PHP as scripting language.
<?php
$exported_objects = array();
foreach ($_MIDGARD['types'] as $type => $type_id) {
$qb = new midgardquerybuilder($type);
$qb->add_constraint("metadata.created", ">", $yesterday);
$retval = $qb->execute();
$exported_objects = array_merge($exported_objects, $retval);
}
foreach($exported_objects as $object){
$object->export();
}
?>
Every Midgard object's metadata class should have 'exported' and 'imported' members assigned as properties of midgard_metadata type. These two properties's values represented by corresponding database storage values should be settable only by midgard-core with particular methods. Both properties must be MGD_TYPE_DATETIME type.
Midgard-core must support export and import methods for object's replication. Object's replication related metadata should be managed by basic methods: create, update, delete, undelete , purge, import, export.
Object can be imported to database only if imported metadata property value of the object for which record in database exists is empty and its metadata revised property value is not newer than the value of the same property of object which is to be imported.
Object can be exported from database without any restriction. Application which exports objects defines what constraints are used to export object records.
This example demonstrates how to export newest and not hidden midgard_article objects
<?php
$qb = new midgardquerybuilder("midgard_article");
$qb->add_constraint("metadata.created", ">", $yesterday);
$qb->add_constraint("metadata.hidden", "=", FALSE);
/* Do not export objects which were created and exported for the last 24 hours */
$qb->add_constraint("metadata.exported", "=", "");
$retval = $qb->execute();
foreach($retval as $object){
$object->export();
}
?>
Object's record(s) can not be deleted from database when delete method is invoked. Instead , object's metadata delete property should be explicitly updated with correct delete value. For performance reason this value should be an integer type. This metadata property should be also used by midgard core as mandatory Midgard Query Builder's constraint added internally by Query Builder implementation.
When this method is invoked, midgard core should delete object's record(s) from database and should update repligard table. Following values should be set for corresponding record in repligard table:
Repligard table must containt only object's guid and typename ( classname ) for which new object instance with particular guid can be created. Additionaly repligard table should contain information if object's record(s) was purged.
Midgard core should implement functionality which allows to query only deleted objects. This functionality could be implemented with new Midgard Query Builder method or with new Midgard type.
Missing topics: Data model, general service architecture, including the layers of the communication infrastucture.
Torben
Since replication interfaces are exposed to PHP level, the staging/live process can be handled in MidCOM space. This should make the simple replication scenarios like an article that is approved much faster, the system more fault tolerant due to the queue system, and finally the replication logic easier to tweak because it is entirely in PHP level.
There will be a midcom.helper.staginglive purecode component to handle the replication process.
This will have to be
midcom.services.replication.Torben
The replication component will register UPDATE and DELETE watchers for midcom_core_dbobject. Since approvals are stored into Midgard metadata fields using update() method approval should be carried to this watcher.
Note, that you cannot register watches for
midcom_core_dbobject, as this class is not a base class for other DBA classes. It is more of a "mix-in" due to Limitations of the PHP OOP API. The correct definition would be adding watches to all defined DBA classes by not limiting the watch to any class.Torben
To catch objects updated or deleted outside MidCOM DBA the system will also provide a cron entry that will query through all MgdSchema types to see if there is something to replicate. This can be run relatively seldom, for example once per day.
This is not very precise. The criteria after which objects are marked for replication should be outlined.
Torben
The export watchers will execute an exporting method in the midcom.helper.staginglive interface class and provide it with the affected object as argument.
If approvals or scheduling are being used, the system will first check if the object itself is set to be visible. Otherwise visibility will be assumed.
If the object is visible, the same check will be done for its parent.
Note, that this can prohibit replication under certain circumstances: Assume that a topic and one of its articles are both changed. The article gets approved, the topic not. Replication of the article will then be delayed.
Note as well, that there is another case, where you actually need to be careful if you change the behavoir to cover the above special case: If the topic in question was newly created, it won't be available on the target server.
Normally, the latter should be no problem if and only if only GUIDs are used for linking and, as such, ID mappings are not required. The legacy data structures do not yet do this. I very strongly suggest that this will be changed prior implementing this, even if this means that existing code (mainly QB lookups) have to be adapted to it.
Torben
If the parent is visible, the object will be marked for exporting. If the parent is not visible export is aborted and UI message "Object not exported because parent is not visible" sent to the user.
What happens later, when the parent gets cleared? Is the delayed object queued somewhere? This needs much more detail.
Torben
Then the method will query the object's children and call the same method for them.
I assume that this is the fix for the above problem I mentioned. The problem I see here is that you will often get large trees here, especially if you are at the top of the tree. Also you will have difficulties as you have to know all types which potentially have the object in question as a parent:
Especially topics are tricky here, as several of the components I wrote lately assign their data to topics, without actually using
midgard_article. But groups or persons can easily reach a similar spread.Torben
TODO: Pseudocode for the exporting decision
I'd like to see this before +1'ing anything.
Torben
Finally the exported objects will be stored into replication queue and an at entry registered for running the replication next minute.
Is this wise? Or would it be more appropriate to have a cron job doing it every X minutes to optimize / batch the changes.
Torben
UI message "3 articles and 1 topic exported for replication" is sent to the user.
Subscribers (Midgard sites to replicate to) are stored in the database. The subscriber entry contains the replication method used for that subscriber and address and authentication information required for the method.
When objects are exported, the export XML is stored into a midcom_helper_staginglive_queue_entry object for each subscriber.
Data duplication, the data model should be revised here.
Torben
Replication is launched by the at entry registered at export phase. The replication process goes through all subscribers, and tries to send the items of their queue to them using the defined method.
There can be multiple replication methods supported by the system. At first we will only implement a simple HTTP PUT (or POST) method sent to a MidCOM exec handler of the midcom.helper.staginglive component, authenticated using HTTP Basic Auth. Other methods could include Jabber/XMPP, DBE or even email.
Wouldn't some XML-RPC or other standardized remoting mechanism be more appropriate then some proprietary solution?
Torben
Email could provide a high security replication solution where the exporting end would GPG encrypt and sign the email, then send it out as an email. The importing end would then read the email from the server, check its signature, decrypt it and import. This way the importing end could even be a mostly offline computer that would only connect to internet to receive the emails once per day (this is how FSF Europe's member registry works to protect privacy).
Consider using regular X.509 based encryption / signing instead of PGP. More standardized (almost as easy to use) and integrated into existing PKI infrastructures. This should be available completly independant of the actual transport layer used, btw.
Torben
If the replication is successful, the queue entry is deleted. If not, the replication will be attempted again every hour using a cron entry.
Provide some fallback mechanism, similar to the SMTP system. There should be a fixed number of retries, maybe with increased waiting periods after certain escalation levels.
Also, it is recommended to flag entries as failed after a certain amount of retries have been reached.
Failed connections to a subscriber should escalate as well, disabling the subscriber until the situation is resolved. Otherwise, there would be an easy DOS scenario when too many replications pile up for a given subscriber.
Torben
In the initial implementation every object is replicated to every subscriber. At a later stage we can implement different subscription type handlers that will enable limiting the subscription to particular MgdSchema types, object trees or even particular query constraints.
Examples of subscription types would be "content in tree", "my tasks" or "all forum posts".
This is rather vague, especially since the neccessary basic infrastructure to provide something like this is never mentioned above. Goes into the chapter "missing service architecture".
Torben
Import process will simply call the Midgard import method for each object in the imported XML file.
]]>No sanity checks, nothing? I think there has to be a high level of bulletproofing when letting such replications in.
Torben
This mRFC is a proposal for repligard functionality replacement , supported by midgard-core and by any language for which Midgard language bindings exist. As repligard supported Midgard database replication without any possibility to change its behaviour a lot , this mRFC focuses on object's records replication , object exporting and object importing to or from any database storage. This mRFC also describes possibility to undelete and purge object's record(s).
Examples in this mRFC use PHP as scripting language.
<?php
$exported_objects = array();
foreach ($_MIDGARD['types'] as $type => $type_id) {
$qb = new midgardquerybuilder($type);
$qb->add_constraint("metadata.created", ">", $yesterday);
$retval = $qb->execute();
$exported_objects = array_merge($exported_objects, $retval);
}
foreach($exported_objects as $object){
$object->export();
}
?>
Every Midgard object's metadata class should have 'exported' and 'imported' members assigned as properties of midgard_metadata type. These two properties's values represented by corresponding database storage values should be settable only by midgard-core with particular methods. Both properties must be MGD_TYPE_DATETIME type.
Midgard-core must support export and import methods for object's replication. Object's replication related metadata should be managed by basic methods: create, update, delete, undelete , purge, import, export.
Object can be imported to database only if imported metadata property value of the object for which record in database exists is empty and its metadata revised property value is not newer than the value of the same property of object which is to be imported.
Object can be exported from database without any restriction. Application which exports objects defines what constraints are used to export object records.
This example demonstrates how to export newest and not hidden midgard_article objects
<?php
$qb = new midgardquerybuilder("midgard_article");
$qb->add_constraint("metadata.created", ">", $yesterday);
$qb->add_constraint("metadata.hidden", "=", FALSE);
/* Do not export objects which were created and exported for the last 24 hours */
$qb->add_constraint("metadata.exported", "=", "");
$retval = $qb->execute();
foreach($retval as $object){
$object->export();
}
?>
Object's record(s) can not be deleted from database when delete method is invoked. Instead , object's metadata delete property should be explicitly updated with correct delete value. For performance reason this value should be an integer type. This metadata property should be also used by midgard core as mandatory Midgard Query Builder's constraint added internally by Query Builder implementation.
When this method is invoked, midgard core should delete object's record(s) from database and should update repligard table. Following values should be set for corresponding record in repligard table:
Repligard table must containt only object's guid and typename ( classname ) for which new object instance with particular guid can be created. Additionaly repligard table should contain information if object's record(s) was purged.
Midgard core should implement functionality which allows to query only deleted objects. This functionality could be implemented with new Midgard Query Builder method or with new Midgard type.
Missing topics: Data model, general service architecture, including the layers of the communication infrastucture.
Torben
Since replication interfaces are exposed to PHP level, the staging/live process can be handled in MidCOM space. This should make the simple replication scenarios like an article that is approved much faster, the system more fault tolerant due to the queue system, and finally the replication logic easier to tweak because it is entirely in PHP level.
There will be a midcom.helper.staginglive purecode component to handle the replication process.
This will have to be
midcom.services.replication.Torben
The replication component will register UPDATE and DELETE watchers for midcom_core_dbobject. Since approvals are stored into Midgard metadata fields using update() method approval should be carried to this watcher.
Note, that you cannot register watches for
midcom_core_dbobject, as this class is not a base class for other DBA classes. It is more of a "mix-in" due to Limitations of the PHP OOP API. The correct definition would be adding watches to all defined DBA classes by not limiting the watch to any class.Torben
To catch objects updated or deleted outside MidCOM DBA the system will also provide a cron entry that will query through all MgdSchema types to see if there is something to replicate. This can be run relatively seldom, for example once per day.
This is not very precise. The criteria after which objects are marked for replication should be outlined.
Torben
The export watchers will execute an exporting method in the midcom.helper.staginglive interface class and provide it with the affected object as argument.
If approvals or scheduling are being used, the system will first check if the object itself is set to be visible. Otherwise visibility will be assumed.
If the object is visible, the same check will be done for its parent.
Note, that this can prohibit replication under certain circumstances: Assume that a topic and one of its articles are both changed. The article gets approved, the topic not. Replication of the article will then be delayed.
Note as well, that there is another case, where you actually need to be careful if you change the behavoir to cover the above special case: If the topic in question was newly created, it won't be available on the target server.
Normally, the latter should be no problem if and only if only GUIDs are used for linking and, as such, ID mappings are not required. The legacy data structures do not yet do this. I very strongly suggest that this will be changed prior implementing this, even if this means that existing code (mainly QB lookups) have to be adapted to it.
Torben
If the parent is visible, the object will be marked for exporting. If the parent is not visible export is aborted and UI message "Object not exported because parent is not visible" sent to the user.
What happens later, when the parent gets cleared? Is the delayed object queued somewhere? This needs much more detail.
Torben
Then the method will query the object's children and call the same method for them.
I assume that this is the fix for the above problem I mentioned. The problem I see here is that you will often get large trees here, especially if you are at the top of the tree. Also you will have difficulties as you have to know all types which potentially have the object in question as a parent:
Especially topics are tricky here, as several of the components I wrote lately assign their data to topics, without actually using
midgard_article. But groups or persons can easily reach a similar spread.Torben
TODO: Pseudocode for the exporting decision
I'd like to see this before +1'ing anything.
Torben
Finally the exported objects will be stored into replication queue and an at entry registered for running the replication next minute.
Is this wise? Or would it be more appropriate to have a cron job doing it every X minutes to optimize / batch the changes.
Torben
UI message "3 articles and 1 topic exported for replication" is sent to the user.
Subscribers (Midgard sites to replicate to) are stored in the database. The subscriber entry contains the replication method used for that subscriber and address and authentication information required for the method.
When objects are exported, the export XML is stored into a midcom_helper_staginglive_queue_entry object for each subscriber.
Data duplication, the data model should be revised here.
Torben
Replication is launched by the at entry registered at export phase. The replication process goes through all subscribers, and tries to send the items of their queue to them using the defined method.
There can be multiple replication methods supported by the system. At first we will only implement a simple HTTP PUT (or POST) method sent to a MidCOM exec handler of the midcom.helper.staginglive component, authenticated using HTTP Basic Auth. Other methods could include Jabber/XMPP, DBE or even email.
Wouldn't some XML-RPC or other standardized remoting mechanism be more appropriate then some proprietary solution?
Torben
Email could provide a high security replication solution where the exporting end would GPG encrypt and sign the email, then send it out as an email. The importing end would then read the email from the server, check its signature, decrypt it and import. This way the importing end could even be a mostly offline computer that would only connect to internet to receive the emails once per day (this is how FSF Europe's member registry works to protect privacy).
Consider using regular X.509 based encryption / signing instead of PGP. More standardized (almost as easy to use) and integrated into existing PKI infrastructures. This should be available completly independant of the actual transport layer used, btw.
Torben
If the replication is successful, the queue entry is deleted. If not, the replication will be attempted again every hour using a cron entry.
Provide some fallback mechanism, similar to the SMTP system. There should be a fixed number of retries, maybe with increased waiting periods after certain escalation levels.
Also, it is recommended to flag entries as failed after a certain amount of retries have been reached.
Failed connections to a subscriber should escalate as well, disabling the subscriber until the situation is resolved. Otherwise, there would be an easy DOS scenario when too many replications pile up for a given subscriber.
Torben
In the initial implementation every object is replicated to every subscriber. At a later stage we can implement different subscription type handlers that will enable limiting the subscription to particular MgdSchema types, object trees or even particular query constraints.
Examples of subscription types would be "content in tree", "my tasks" or "all forum posts".
This is rather vague, especially since the neccessary basic infrastructure to provide something like this is never mentioned above. Goes into the chapter "missing service architecture".
Torben
Import process will simply call the Midgard import method for each object in the imported XML file.
]]>No sanity checks, nothing? I think there has to be a high level of bulletproofing when letting such replications in.
Torben