PHP's lazy loading and the use of Iterator

sponsored links
Recently changed jobs, changed into the software establishment of the station, and our establishment of the station software is a problem with the template system, suggesting that the memory beyond the maximum. Memory problems are most simple way is to modify the php.ini to improve memory_limit, but with the increasing number of template required memory goes beyond the upper limit, so the boss let me take a look at this issue, look at me what a good way.

I got this problem, first read the original code, ideas, analysis of the main causes of the problem, problem page is simply a template display page 16 per page templates, to do page processing, and the background treatment is not simple, the first template data is not read from the database, but from the server to obtain a serialized string, remove all of the template is an array of information, information is then cached (after the first reading of the cache) and then array to filter, and paging operations, remove the template display 16.

Then analyze the problem, it is clear where the problem is with all of the templates out of the array of information on this issue is a huge number of templates, but in fact only 16 per needs only, but here it all added to achieve memory.

Problems found, how to solve it? So I asked the former employees, listen to their views. They provide a lot of views dealing with the template into the database, paging and other operations have to put all of our servers processing, but programs need to expand our back-end server processing improvements and changes things too much, the problem of the introduction of more, how do?

1. Lazy loading

Then I thought, since the problem lies in removing all the serialized data, if we can in the file, just remove the data we need good, because I have java experience, I first thought was XML SAX processing methods, But I'm not familiar with PHP, XML processing, to no avail.

But the right gave me the idea, a lot of PHP memory is too large problems due to PHP developers for the convenience, all required data pre-loaded into memory (array), then by addressing such issues is the result of memory required far exceeded its actual needs, has led to excessive memory problems. The solution is lazy loading, so that the use of data loaded into memory when processing.

The problem here is clearly to the issue of sequence, PHP's serialize mechanism is a strategy of pre-loaded, does not support lazy loading. Need to find a support serialization of lazy loading approach, a template can be a time to read the contents of treatment, and soon appeared in CSV format, CSV to support this approach to read in units, and for the original system mechanism does not require any major changes, so will the original system csv serialization changes.

2.Iterator interface uses

csv use while reading to traverse, while the original system uses an array to traverse, the system code filter conditions, involving a lot of code changes, and the previous array using the key => value pairs to achieve, and csv can only use 0, 1,2 digital access. To minimize the original code changes, we first made a deal on the csv, it's the first line array of key values, starting from the second line shows the actual value of the template value, location and key correspondence, which also meets the future scalability.

Foreach order and style of the original code as far as possible match, we use the operation of Iterator encapsulates the csv and csv completed inside the key-> value of the merger. Thus, for code migration is almost negligible, the greater the advantage of csv file size than the file size of the serialized under a lot of, some people may worry about performance issues.

After testing our existing template 2000, the speed does not exceed 1 second, the next number to be increased to 20,000 templates, traverse speed of no more than 2s, are also acceptable range. So the problem is solved.

Recently, we made a simple ORM framework of a memory exceeds the maximum, also owing to the value of all the databases into the memory array, so I lazy load the spear in one hand, shield in one hand Iterator once again into solve the problem of war.
  • del.icio.us
  • StumbleUpon
  • Digg
  • TwitThis
  • Mixx
  • Technorati
  • Facebook
  • NewsVine
  • Reddit
  • Google
  • LinkedIn
  • YahooMyWeb

Related Posts of PHP's lazy loading and the use of Iterator

  • An essential reference resource list of Java

    Since the Java platform since 1995 as a whole began to introduce programming community, and its development has gone far beyond the early Java experts and those who promote the idea of "applet ubiquitous" of that vision. In contrast, Java emerge

  • Eclipse to run using the specified JVM m2eclipse plugin can not find tools.jar

    Used the m2eclipse plug-ins using struts2 when com.sun necessary to rely on the default-tools.jar, specifically because at the struts-annotations bag designated default-tools.jar <profile> <id> default-tools.jar </ id> <activatio ...

  • Verification Code Rails implementation continued

    http://sanyking.javaeye.com/blog/120339 Connect with the address above on this website are one of the other way regarding the implementation Verification Code article, my article are based on the above completed, and the world is one big copy article, loo

  • Play flash page 3 (javascript script swfobject.js1)

    / ** * SWFObject v1.5: Flash Player detection and embed - http://blog.deconcept.com/swfobject/ * * SWFObject is (c) 2007 Geoff Stearns and is released under the MIT License: * Http://www.opensource.org/licenses/mit-license.php * * / if (typeof deconc ...

  • javascript built-in functions Info

    48. In the old browsers do not perform this JS :<!-- //--> 49. To quote a document-style JS: <script type="text/javascript" src="/blog/aaa.js"> </ script> 50. Specified in the script does not support the browser displ

  • Imitation Java type HashMap implementation of JavaScript

    Imitation Java type HashMap implementation of JavaScript

  • haproxy should not keep an accurate record of the client IP,

    Apache + HAProxy a server (A) Rails App two servers (B, C) version = 2.0.4 Medium production.log suddenly discovered some time ago only shows haproxy of IP. And should not show the real client ip However, in / etc / haproxy.cfg has been annotated Ini ...

  • Javascript object inheritance

    Javascript object inheritance

  • Javascript Object Model

    Javascript Object Model

  • Hibernate II Study Notes

    11. Many-to-many Of many that can be converted to two one-to-many <set name="students" table="teacher_student"> <key column="techer_id"/> <many-to-many column="student_id"/> </set> many-to-many data only from one end of the mainten

blog comments powered by Disqus
Recent
Recent Entries
Tag Cloud
Random Entries