PHP's lazy loading and the use of Iterator

Recently changed jobs, changed into the software establishment of the station, and our establishment of the station software is a problem with the template system, suggesting that the memory beyond the maximum. Memory problems are most simple way is to modify the php.ini to improve memory_limit, but with the increasing number of template required memory goes beyond the upper limit, so the boss let me take a look at this issue, look at me what a good way.

I got this problem, first read the original code, ideas, analysis of the main causes of the problem, problem page is simply a template display page 16 per page templates, to do page processing, and the background treatment is not simple, the first template data is not read from the database, but from the server to obtain a serialized string, remove all of the template is an array of information, information is then cached (after the first reading of the cache) and then array to filter, and paging operations, remove the template display 16.

Then analyze the problem, it is clear where the problem is with all of the templates out of the array of information on this issue is a huge number of templates, but in fact only 16 per needs only, but here it all added to achieve memory.

Problems found, how to solve it? So I asked the former employees, listen to their views. They provide a lot of views dealing with the template into the database, paging and other operations have to put all of our servers processing, but programs need to expand our back-end server processing improvements and changes things too much, the problem of the introduction of more, how do?

1. Lazy loading

Then I thought, since the problem lies in removing all the serialized data, if we can in the file, just remove the data we need good, because I have java experience, I first thought was XML SAX processing methods, But I'm not familiar with PHP, XML processing, to no avail.

But the right gave me the idea, a lot of PHP memory is too large problems due to PHP developers for the convenience, all required data pre-loaded into memory (array), then by addressing such issues is the result of memory required far exceeded its actual needs, has led to excessive memory problems. The solution is lazy loading, so that the use of data loaded into memory when processing.

The problem here is clearly to the issue of sequence, PHP's serialize mechanism is a strategy of pre-loaded, does not support lazy loading. Need to find a support serialization of lazy loading approach, a template can be a time to read the contents of treatment, and soon appeared in CSV format, CSV to support this approach to read in units, and for the original system mechanism does not require any major changes, so will the original system csv serialization changes.

2.Iterator interface uses

csv use while reading to traverse, while the original system uses an array to traverse, the system code filter conditions, involving a lot of code changes, and the previous array using the key => value pairs to achieve, and csv can only use 0, 1,2 digital access. To minimize the original code changes, we first made a deal on the csv, it's the first line array of key values, starting from the second line shows the actual value of the template value, location and key correspondence, which also meets the future scalability.

Foreach order and style of the original code as far as possible match, we use the operation of Iterator encapsulates the csv and csv completed inside the key-> value of the merger. Thus, for code migration is almost negligible, the greater the advantage of csv file size than the file size of the serialized under a lot of, some people may worry about performance issues.

After testing our existing template 2000, the speed does not exceed 1 second, the next number to be increased to 20,000 templates, traverse speed of no more than 2s, are also acceptable range. So the problem is solved.

Recently, we made a simple ORM framework of a memory exceeds the maximum, also owing to the value of all the databases into the memory array, so I lazy load the spear in one hand, shield in one hand Iterator once again into solve the problem of war.

分类:PHP 时间:2011-05-23 人气:138
blog comments powered by Disqus


  • Part I. Designed Quick Start Guide --- --- Template + data model = output 2009-03-27

    Chapter 1. Quick Start Quick Start Template + data model = output Template + data model = output FreeMarker is based on an idea: designers and developers are different people, they were good at different skills. Ideally, designers should focus attent

  • ASP.NET 2.0 to implement data binding template 2010-07-07

    Templated data-bound controls to display data on the page we provide a fundamental flexibility. You may recall ASP.NET v1.x Several template-based controls (such as DataList, and Repeater controls). ASP.NET 2.0 continues to support these controls, bu

  • Heyes Template Class php template engine 2010-09-11

    Program Name: Heyes Template Class php template engine developer / Organization Name: Richard Heyes Operating environment: php Official Website: Download: / template / Source: http://www.osphp.c

  • C + + class template in the Template can be used when 2010-11-02

    Multiple classes share a common operation, but the data type. The following three categories, getMax function is the same, that is seeking the maximum of two numbers, only the data type. class Compare_int { private: int x,y; public: Compare(int a,int

  • Universal Data Access Control System Design 2009-02-28

    Figure II: general data rights management system database design Comparison of two maps, we can see that the main changes between them are: 1, increase system resources, information and operation type information, the system resources for the tree st

  • Universal Data Access Control System (Reprinted) 2010-06-20 Universal Data Access Control System Analysis: Yi Yun Source: Internet Introduction: This article provides an integrated function of rights and data rights solutions to meet the multi-level rights management o

  • Using memcached shared between multiple servers in PHP's session data 2010-06-08

    Recently, my articles are a bit of recent project experience ^ ^ The project started on the design of the "great", considering a lot of load balancing side of things, such as using nginx to php requests shared by a number of independent running

  • August 27, 2010 --- RedDwarf as data communication between server and when the byte array type to other types of data conversion 2010-08-27

    Engage in this because: RedDwarf way server platform is based on byte array of data types, so need to do a byte array to convert basic data types. The main technology used MINA and AMF, the first to introduce the AMF: Action Message Format (AMF) is a

  • Type array, javascript way to access binary data 2011-05-25

    Know that javascript can access binary code This specification provides an API for interoperability with native binary data. It defines a generic fixed-length buffer type, as well as accessor types that allow access to the data stored within the buff

  • JAVA operating word reading template files 2010-03-23

    Want to do a java action word of the program, to achieve reading the template file, modify the part, generate a new file such a function. package zb; import; import; import; import

iOS 开发

Android 开发

Python 开发



PHP 开发

Ruby 开发






Javascript 开发

.NET 开发



Copyright (C), All Rights Reserved. 版权所有 闽ICP备15018612号

processed in 0.057 (s). 13 q(s)