I got this problem, first read the original code, ideas, analysis of the main causes of the problem, problem page is simply a template display page 16 per page templates, to do page processing, and the background treatment is not simple, the first template data is not read from the database, but from the server to obtain a serialized string, remove all of the template is an array of information, information is then cached (after the first reading of the cache) and then array to filter, and paging operations, remove the template display 16.
Then analyze the problem, it is clear where the problem is with all of the templates out of the array of information on this issue is a huge number of templates, but in fact only 16 per needs only, but here it all added to achieve memory.
Problems found, how to solve it? So I asked the former employees, listen to their views. They provide a lot of views dealing with the template into the database, paging and other operations have to put all of our servers processing, but programs need to expand our back-end server processing improvements and changes things too much, the problem of the introduction of more, how do?
1. Lazy loading
Then I thought, since the problem lies in removing all the serialized data, if we can in the file, just remove the data we need good, because I have java experience, I first thought was XML SAX processing methods, But I'm not familiar with PHP, XML processing, to no avail.
But the right gave me the idea, a lot of PHP memory is too large problems due to PHP developers for the convenience, all required data pre-loaded into memory (array), then by addressing such issues is the result of memory required far exceeded its actual needs, has led to excessive memory problems. The solution is lazy loading, so that the use of data loaded into memory when processing.
The problem here is clearly to the issue of sequence, PHP's serialize mechanism is a strategy of pre-loaded, does not support lazy loading. Need to find a support serialization of lazy loading approach, a template can be a time to read the contents of treatment, and soon appeared in CSV format, CSV to support this approach to read in units, and for the original system mechanism does not require any major changes, so will the original system csv serialization changes.
2.Iterator interface uses
csv use while reading to traverse, while the original system uses an array to traverse, the system code filter conditions, involving a lot of code changes, and the previous array using the key => value pairs to achieve, and csv can only use 0, 1,2 digital access. To minimize the original code changes, we first made a deal on the csv, it's the first line array of key values, starting from the second line shows the actual value of the template value, location and key correspondence, which also meets the future scalability.
Foreach order and style of the original code as far as possible match, we use the operation of Iterator encapsulates the csv and csv completed inside the key-> value of the merger. Thus, for code migration is almost negligible, the greater the advantage of csv file size than the file size of the serialized under a lot of, some people may worry about performance issues.
After testing our existing template 2000, the speed does not exceed 1 second, the next number to be increased to 20,000 templates, traverse speed of no more than 2s, are also acceptable range. So the problem is solved.
Recently, we made a simple ORM framework of a memory exceeds the maximum, also owing to the value of all the databases into the memory array, so I lazy load the spear in one hand, shield in one hand Iterator once again into solve the problem of war.
Related Posts of PHP's lazy loading and the use of Iterator
Since the Java platform since 1995 as a whole began to introduce programming community, and its development has gone far beyond the early Java experts and those who promote the idea of "applet ubiquitous" of that vision. In contrast, Java emerge
Used the m2eclipse plug-ins using struts2 when com.sun necessary to rely on the default-tools.jar, specifically because at the struts-annotations bag designated default-tools.jar <profile> <id> default-tools.jar </ id> <activatio ...
http://sanyking.javaeye.com/blog/120339 Connect with the address above on this website are one of the other way regarding the implementation Verification Code article, my article are based on the above completed, and the world is one big copy article, loo
/ ** * SWFObject v1.5: Flash Player detection and embed - http://blog.deconcept.com/swfobject/ * * SWFObject is (c) 2007 Geoff Stearns and is released under the MIT License: * Http://www.opensource.org/licenses/mit-license.php * * / if (typeof deconc ...
Apache + HAProxy a server (A) Rails App two servers (B, C) version = 2.0.4 Medium production.log suddenly discovered some time ago only shows haproxy of IP. And should not show the real client ip However, in / etc / haproxy.cfg has been annotated Ini ...
11. Many-to-many Of many that can be converted to two one-to-many <set name="students" table="teacher_student"> <key column="techer_id"/> <many-to-many column="student_id"/> </set> many-to-many data only from one end of the mainten