For a more efficient MidCOM 2.9
-
Henri Bergius
For a more efficient MidCOM 2.9
Mon August 11 2008 15:47:40 UTCHi, all
According to our new strategy, Midgard 1.9 ("Ragnaroek") will be a
long-term supported release. This is mainly because in Ragnaroek+1 we
will jump to MidCOM3.
So this means we will be living with the older MidCOM generation for
years still (as there are lots of big sites running on it). And that
means we should tweak it so that we can actually live with it.
Main points:
* Implement autoloading and magic getters to prevent loading of
unnecessary PHP files
* Switch from Prototype/Scriptaculous to jQuery (probably can't be
done completely, but the components still not ported should be noted
and isolated)
* Clean up autoload_files and autoload_libraries of components
Of course the structure of MidCOM 2 being what it is, we cannot make
miracles. But we should be able to make substantial advances in these
areas. As requested by Niels, I spent much of today implementing
autoloading into MidCOM 2.9, and the results were quite promising:
* Phase 1 (no autoloading): 252 included files
* Phase 2 (some autoloading): 216 included files
* Phase 3 (Some component cleanup): 175 included files
* Phase 4 (DBAs loaded only when component is): 129 included files
* Phase 5 (service autoloading): 124 included files
All of these were hitting a n.n.static page with authenticated user,
AJAX editor, toolbars etc. With anonymous user, the final number was
105 included files. And it feels a lot more snappy too.
105 might still sound a lot, but you have to consider this is system
running with full ACLs, sessioning, i18n, caching, etc. Midgard is a
more "serious CMS", and lots of functionality requires lots of PHP
classes. But obviously for MidCOM 3 this will be better as there will
be no DBA etc.
I've implemented this now in a private git branch, but as the results
are very good, I think I should merge it into MidCOM trunk still
today. Please let me know if you encounter any issues.
We can still get better results with the autoloader. How you can help
is by going through components, and check for the following things:
* autoload_files: all of these pointing to class can be safely
removed. those pointing to helper function files must stay
* autoload_libraries: check if the library could be loaded only in the
handlers that actually use it
* include, require, require_once: check if these are really necessary,
and run only where really needed
All OpenPsa2 libraries are especially a dependency mess, loading some
100 unnecessary PHP files. Would be great to have it cleaned up.
/Bergie
--
Henri Bergius
Motorcycle Adventures and Free Software
http://bergie.iki.fi/
Skype: henribergius
Jabber: henri.bergius@gmail.com
Jaiku: http://bergie.jaiku.com/
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] For a more efficient MidCOM 2.9
Wed August 13 2008 18:27:35 UTCHi!
I just played around with removing some autoload_files from openpsa
components, but every time I remove one file, the number of included
files is one higher. This happens because each file is listed twice:
/usr/share/php5/PEAR/midcom/lib/midcom/baseclasses/database/eventmember.php
/var/svn_checkout/trunk/midcom/midcom.core/midcom/baseclasses/database/eventmember.php
The first path is a symlink, the second path is the real file in the svn
checkout. Does the file really get loaded twice or is this just some
problem with get_included_files()?
Bye,
Andreas
Henri Bergius schrieb:
> Hi, all
>
> According to our new strategy, Midgard 1.9 ("Ragnaroek") will be a
> long-term supported release. This is mainly because in Ragnaroek+1 we
> will jump to MidCOM3.
>
> So this means we will be living with the older MidCOM generation for
> years still (as there are lots of big sites running on it). And that
> means we should tweak it so that we can actually live with it.
>
> Main points:
> * Implement autoloading and magic getters to prevent loading of
> unnecessary PHP files
> * Switch from Prototype/Scriptaculous to jQuery (probably can't be
> done completely, but the components still not ported should be noted
> and isolated)
> * Clean up autoload_files and autoload_libraries of components
>
> Of course the structure of MidCOM 2 being what it is, we cannot make
> miracles. But we should be able to make substantial advances in these
> areas. As requested by Niels, I spent much of today implementing
> autoloading into MidCOM 2.9, and the results were quite promising:
>
> * Phase 1 (no autoloading): 252 included files
> * Phase 2 (some autoloading): 216 included files
> * Phase 3 (Some component cleanup): 175 included files
> * Phase 4 (DBAs loaded only when component is): 129 included files
> * Phase 5 (service autoloading): 124 included files
>
> All of these were hitting a n.n.static page with authenticated user,
> AJAX editor, toolbars etc. With anonymous user, the final number was
> 105 included files. And it feels a lot more snappy too.
>
> 105 might still sound a lot, but you have to consider this is system
> running with full ACLs, sessioning, i18n, caching, etc. Midgard is a
> more "serious CMS", and lots of functionality requires lots of PHP
> classes. But obviously for MidCOM 3 this will be better as there will
> be no DBA etc.
>
> I've implemented this now in a private git branch, but as the results
> are very good, I think I should merge it into MidCOM trunk still
> today. Please let me know if you encounter any issues.
>
> We can still get better results with the autoloader. How you can help
> is by going through components, and check for the following things:
>
> * autoload_files: all of these pointing to class can be safely
> removed. those pointing to helper function files must stay
> * autoload_libraries: check if the library could be loaded only in the
> handlers that actually use it
> * include, require, require_once: check if these are really necessary,
> and run only where really needed
>
> All OpenPsa2 libraries are especially a dependency mess, loading some
> 100 unnecessary PHP files. Would be great to have it cleaned up.
>
> /Bergie
>
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] For a more efficient MidCOM 2.9
Thu August 14 2008 07:40:07 UTCHi!
Just a small addition: When I change line 156 of midcom php from require
to require_once, the included file counts drops quite a bit, f.x. from
280 to 194 on the front page of OpenPSA.
Bye,
Andreas
Andreas Flack schrieb:
> Hi!
>
> I just played around with removing some autoload_files from openpsa
> components, but every time I remove one file, the number of included
> files is one higher. This happens because each file is listed twice:
>
> /usr/share/php5/PEAR/midcom/lib/midcom/baseclasses/database/eventmember.php
> /var/svn_checkout/trunk/midcom/midcom.core/midcom/baseclasses/database/eventmember.php
>
>
>
> The first path is a symlink, the second path is the real file in the svn
> checkout. Does the file really get loaded twice or is this just some
> problem with get_included_files()?
>
>
>
> Bye,
>
> Andreas
>
> Henri Bergius schrieb:
>> Hi, all
>>
>> According to our new strategy, Midgard 1.9 ("Ragnaroek") will be a
>> long-term supported release. This is mainly because in Ragnaroek+1 we
>> will jump to MidCOM3.
>>
>> So this means we will be living with the older MidCOM generation for
>> years still (as there are lots of big sites running on it). And that
>> means we should tweak it so that we can actually live with it.
>>
>> Main points:
>> * Implement autoloading and magic getters to prevent loading of
>> unnecessary PHP files
>> * Switch from Prototype/Scriptaculous to jQuery (probably can't be
>> done completely, but the components still not ported should be noted
>> and isolated)
>> * Clean up autoload_files and autoload_libraries of components
>>
>> Of course the structure of MidCOM 2 being what it is, we cannot make
>> miracles. But we should be able to make substantial advances in these
>> areas. As requested by Niels, I spent much of today implementing
>> autoloading into MidCOM 2.9, and the results were quite promising:
>>
>> * Phase 1 (no autoloading): 252 included files
>> * Phase 2 (some autoloading): 216 included files
>> * Phase 3 (Some component cleanup): 175 included files
>> * Phase 4 (DBAs loaded only when component is): 129 included files
>> * Phase 5 (service autoloading): 124 included files
>>
>> All of these were hitting a n.n.static page with authenticated user,
>> AJAX editor, toolbars etc. With anonymous user, the final number was
>> 105 included files. And it feels a lot more snappy too.
>>
>> 105 might still sound a lot, but you have to consider this is system
>> running with full ACLs, sessioning, i18n, caching, etc. Midgard is a
>> more "serious CMS", and lots of functionality requires lots of PHP
>> classes. But obviously for MidCOM 3 this will be better as there will
>> be no DBA etc.
>>
>> I've implemented this now in a private git branch, but as the results
>> are very good, I think I should merge it into MidCOM trunk still
>> today. Please let me know if you encounter any issues.
>>
>> We can still get better results with the autoloader. How you can help
>> is by going through components, and check for the following things:
>>
>> * autoload_files: all of these pointing to class can be safely
>> removed. those pointing to helper function files must stay
>> * autoload_libraries: check if the library could be loaded only in the
>> handlers that actually use it
>> * include, require, require_once: check if these are really necessary,
>> and run only where really needed
>>
>> All OpenPsa2 libraries are especially a dependency mess, loading some
>> 100 unnecessary PHP files. Would be great to have it cleaned up.
>>
>> /Bergie
>>
>
> _______________________________________________
> dev mailing list
> dev@lists.midgard-project.org
> http://lists.midgard-project.org/mailman/listinfo/dev
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] For a more efficient MidCOM 2.9
Thu August 14 2008 07:43:52 UTCHi,
On Thu, Aug 14, 2008 at 10:40 AM, Andreas Flack
<flack@contentcontrol-berlin.de> wrote:
>> I just played around with removing some autoload_files from openpsa
>> components, but every time I remove one file, the number of included files
>> is one higher. This happens because each file is listed twice:
>> /usr/share/php5/PEAR/midcom/lib/midcom/baseclasses/database/eventmember.php
>> /var/svn_checkout/trunk/midcom/midcom.core/midcom/baseclasses/database/eventmember.php
>
> Just a small addition: When I change line 156 of midcom php from require to
> require_once, the included file counts drops quite a bit, f.x. from 280 to
> 194 on the front page of OpenPSA.
Sounds like a difference between how OS X and Linux deal with
symlinks, as I don't see the same thing here.
But good that require_once helped. It is a bit inefficient I've
understood, so we have to think if we can somehow reduce the
duplicates.
Do you notice any improvements in performance?
--
Henri Bergius
Motorcycle Adventures and Free Software
http://bergie.iki.fi/
Skype: henribergius
Jabber: henri.bergius@gmail.com
Jaiku: http://bergie.jaiku.com/
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] For a more efficient MidCOM 2.9
Thu August 14 2008 07:48:03 UTCOn Thu, Aug 14, 2008 at 11:43, Henri Bergius <henri.bergius@iki.fi> wrote:
> Hi,
>
> On Thu, Aug 14, 2008 at 10:40 AM, Andreas Flack
> <flack@contentcontrol-berlin.de> wrote:
> >> I just played around with removing some autoload_files from openpsa
> >> components, but every time I remove one file, the number of included
> files
> >> is one higher. This happens because each file is listed twice:
> >>
> /usr/share/php5/PEAR/midcom/lib/midcom/baseclasses/database/eventmember.php
> >>
> /var/svn_checkout/trunk/midcom/midcom.core/midcom/baseclasses/database/eventmember.php
> >
> > Just a small addition: When I change line 156 of midcom php from require
> to
> > require_once, the included file counts drops quite a bit, f.x. from 280
> to
> > 194 on the front page of OpenPSA.
>
> Sounds like a difference between how OS X and Linux deal with
> symlinks, as I don't see the same thing here.
>
> But good that require_once helped. It is a bit inefficient I've
> understood, so we have to think if we can somehow reduce the
> duplicates.
>
> Do you notice any improvements in performance?
>
I need to dig through my old experiments where I made a hook in require()
processing within midgard's PHP module to allow storing content of .php
files in a memory cache. I remember getting much faster loads. When such
code could be put into memcached/any other shared storage, this could allow
for much faster load times as no file system operations would be needed for
most of requests.
This, of course, is good for production sites only, where .php files aren't
updated frequently.
--
/ Alexander Bokovoy -
-
Re: [midgard-dev] For a more efficient MidCOM 2.9
Thu August 14 2008 07:55:17 UTCOn 14.8.2008, at 10.48, Alexander Bokovoy wrote:
>
> Do you notice any improvements in performance?
> I need to dig through my old experiments where I made a hook in
> require() processing within midgard's PHP module to allow storing
> content of .php files in a memory cache. I remember getting much
> faster loads. When such code could be put into memcached/any other
> shared storage, this could allow for much faster load times as no
> file system operations would be needed for most of requests.
>
> This, of course, is good for production sites only, where .php
> files aren't updated frequently.
>
What does the PHP bytecode caches like APC or eAccelerator do? Do
they provide similar functionality or could the files compiled by
those stored in some sort of cache in order to get even more
performance?
- Tero
--
Tero Heikkinen
http://teroheikkinen.iki.fi
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] For a more efficient MidCOM 2.9
Thu August 14 2008 08:42:49 UTCAlexander Bokovoy writes:
Hi!
>> Do you notice any improvements in performance?
>>
> I need to dig through my old experiments where I made a hook in require()
> processing within midgard's PHP module to allow storing content of .php
> files in a memory cache. I remember getting much faster loads. When such
> code could be put into memcached/any other shared storage, this could allow
> for much faster load times as no file system operations would be needed for
> most of requests.
>
> This, of course, is good for production sites only, where .php files aren't
> updated frequently.
It could be easily done with built in cache class and few methods which
should be responsible for:
* caching or not
* invalidating cache
* code execution
Cons:
* we can not use require and similiar functions in code
Pros:
* faster than memcache
* fully customizable for development and production sites
* very flexible for individual files caching
Piotras
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] For a more efficient MidCOM 2.9
Thu August 14 2008 09:00:27 UTCHenri Bergius schrieb:
> Hi,
>
> On Thu, Aug 14, 2008 at 10:40 AM, Andreas Flack
> <flack@contentcontrol-berlin.de> wrote:
>>> I just played around with removing some autoload_files from openpsa
>>> components, but every time I remove one file, the number of included files
>>> is one higher. This happens because each file is listed twice:
>>> /usr/share/php5/PEAR/midcom/lib/midcom/baseclasses/database/eventmember.php
>>> /var/svn_checkout/trunk/midcom/midcom.core/midcom/baseclasses/database/eventmember.php
>> Just a small addition: When I change line 156 of midcom php from require to
>> require_once, the included file counts drops quite a bit, f.x. from 280 to
>> 194 on the front page of OpenPSA.
>
> Sounds like a difference between how OS X and Linux deal with
> symlinks, as I don't see the same thing here.
>
> But good that require_once helped. It is a bit inefficient I've
> understood, so we have to think if we can somehow reduce the
> duplicates.
>
> Do you notice any improvements in performance?
>
I don't have any benchmarking tools set up right now, but when when I
compare it with a stop-watch, I get roughly 14s with require_once and
roughly 16s with require. 2.8/1.8 on a similar system loads the same
page in 21s, so overall the auotoloader seems to help quite a bit.
Bye,
Andreas
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
-
Re: [midgard-dev] For a more efficient MidCOM 2.9
Thu August 14 2008 09:48:16 UTCOn Thu, Aug 14, 2008 at 12:42, Piotr Pokora <piotrek.pokora@gmail.com>wrote:
> Alexander Bokovoy writes:
>
> Hi!
>
> >> Do you notice any improvements in performance?
> >>
> > I need to dig through my old experiments where I made a hook in require()
> > processing within midgard's PHP module to allow storing content of .php
> > files in a memory cache. I remember getting much faster loads. When such
> > code could be put into memcached/any other shared storage, this could
> allow
> > for much faster load times as no file system operations would be needed
> for
> > most of requests.
> >
> > This, of course, is good for production sites only, where .php files
> aren't
> > updated frequently.
>
> It could be easily done with built in cache class and few methods which
> should be responsible for:
>
> * caching or not
> * invalidating cache
> * code execution
>
> Cons:
>
> * we can not use require and similiar functions in code
>
My code was completely transparent to any PHP level code. "It just works"
:-)
--
/ Alexander Bokovoy -
-
Re: [midgard-dev] For a more efficient MidCOM 2.9
Thu August 14 2008 10:00:58 UTCAlexander Bokovoy writes:
Hi!
>> Cons:
>>
>> * we can not use require and similiar functions in code
>>
> My code was completely transparent to any PHP level code. "It just works"
> :-)
You mean you use ZE API hooks for include and eval?
Piotras
_______________________________________________
dev mailing list
dev@lists.midgard-project.org
http://lists.midgard-project.org/mailman/listinfo/dev -
