Rails to monitor the process of memory leaks skills

Rails applications easier to encounter two types of performance problems: Rails implementation of a class are very slow, CPU consumption too high; The other is the process of memory leaks Rails. To resolve these two types of questions are necessary before you start you can pinpoint problems with the code, then we know how to prescribe the right medicine.

First, how the process of monitoring the implementation of the performance of Rails

Positioning high CPU consumption, the slow pace of implementation of the Rails code, is a very easy thing, you just need to do a little statistical analysis production.log, extracted the longest execution time of the request, the problem is evident. Production.log on Rails because of the execution of the request time to do the detailed statistics, such as:

Completed in 0.00693 (144 reqs/sec) | Rendering: 0.00489 (70%) | DB: 0.00000 (0%) | 200 OK [http://www.javaeye.com/]
Completed in 0.17238 (5 reqs/sec) | Rendering: 0.10011 (58%) | DB: 0.06244 (36%) | 200 OK [http://www.javaeye.com/topic/49441?page=7]
Completed in 0.20508 (4 reqs/sec) | Rendering: 0.19373 (94%) | DB: 0.00645 (3%) | 200 OK [http://www.javaeye.com/news/1586]


So we need to do is write a command line shell, they get! Him put the most time-consuming pre-screening of 500 requests out, save it to timing.log inside.

grep "200 OK" production.log | awk '{print "ALL: " $3 "  View: " $8 " DB: " $12 "  URL: " $17 }' \
| sort -r | head -n 500 > timing.log


Best to sort the results such as:

ALL: 5.51774  View: 5.38277 DB: 0.13338  URL: [http://www.javaeye.com/wiki/topic/131966]
ALL: 5.51316  View: 5.31300 DB: 0.19400  URL: [http://www.javaeye.com/wiki/topic/145383]
ALL: 5.51311  View: 5.39321 DB: 0.11234  URL: [http://www.javaeye.com/wiki/topic/160370]
ALL: 5.51135  View: 5.37604 DB: 0.12652  URL: [http://www.javaeye.com/wiki/topic/233365]
ALL: 5.49881  View: 5.35998 DB: 0.10637  URL: [http://www.javaeye.com/wiki/topic/265217]


Which requested the Executive of the slow, at a glance. In addition of course, we can real-time monitoring, surveillance in the top window displays the status of ongoing Rails request URL.

Second, how to monitor process memory leak Rails

CPU monitoring is a very easy thing, but the process of monitoring Rails memory leak is very difficult because there has not production.log Record changes in the process of memory, and even you can not find any ruby API can be used to direct inquiries to the process of physical memory used. In fact, it is necessary to obtain a process of physical memory are related to the operation of a platform, every operating system will be its own specific API, not generic, that is, the use of C language code, nor is it easy.

But for the Linux operating system, we have a shortcut to access the memory state of the process. Linux's / proc file system are kernel image, / proc / process pid / status file records the process status information, such as:
Name:   dispatch.fcgi
State:  S (sleeping)
SleepAVG:       135%
Tgid:   26645
Pid:    26645
PPid:   1
TracerPid:      0
Uid:    1002    1002    1002    1002
Gid:    100     100     100     100
FDSize: 64
Groups: 14 16 17 33 100 
VmSize:   245680 kB
VmLck:         0 kB
VmRSS:    209104 kB
VmData:   205116 kB
VmStk:       824 kB
VmExe:       764 kB
VmLib:      4220 kB
Threads:        1
SigPnd: 0000000000000000
ShdPnd: 0000000000000000
SigBlk: 0000000000000000
SigIgn: 0000000000001000
SigCgt: 0000000002006e47
CapInh: 0000000000000000
CapPrm: 0000000000000000
CapEff: 0000000000000000


Attention to section 14 line VmRSS, Record of the process of physical memory used by the Permanent (Residence), this is actually occupied by the process of physical memory. So long as we read the first 14 lines of the document, you can get memory information.

Therefore become the U.S. mission: before proceeding with the request at Rails Record memory, such as Rails request processing, then record memory, the calculation of changes in memory, write to production.log go inside. Complete this work, we only need to Rails applications at app / controllers / application.rb inside to add a few lines of code:

  around_filter :record_memory
  def record_memory
    process_status = File.open("/proc/#{Process.pid}/status")
    13.times { process_status.gets }
    rss_before_action = process_status.gets.split[1].to_i
    process_status.close
    yield
    process_status = File.open("/proc/#{Process.pid}/status")
    13.times { process_status.gets }
    rss_after_action = process_status.gets.split[1].to_i
    process_status.close
    logger.info("CONSUME MEMORY: #{rss_after_action - rss_before_action} \
KB\tNow: #{rss_after_action} KB\t#{request.url}")
  end


We define a AroundFilter, click Record deal with the request before and after changes in memory. With this information, we are the next things simple, just inside production.log extracted from this line of log, for statistical analysis can be, and this was only his party is only required to get a shell:

grep "CONSUME MEMORY" production.log | grep -v "CONSUME MEMORY: 0" |  \
 grep -v "CONSUME MEMORY: -" |  awk '{print $3 "\t" $6 "\t" $8 }' | sort -r -n | \
 head -n 500 > memory.log 


Record taken from memory, remove the memory does not increase, remove the memory reduction (taken place in GC) the request, and then, following a request to those who deal with the increase in memory records sorted out before the 500 record-keeping to the inside memory.log The results are as follows follows:

 Increase the number of memory footprint request URL  
-----------------------------------------------
9528  175264  http://www.javaeye.com/topic/304594
9524  129512  http://knityster.javaeye.com/blog/172990
9496  147544  http://www.javaeye.com/forums/
9492  197800  http://duyiwuer.javaeye.com/rss
9452  146668  http://www.javaeye.com/forums
9452  133844  http://wildlife.javaeye.com/blog/47693
9440  157824  http://www.javaeye.com/rss/blogs
9424  204664  http://www.javaeye.com/wiki/topic/251964
9384  142200  http://towerhe.javaeye.com/blog/93704
9380  165372  http://www.javaeye.com/wiki/topic/77434
9368  207460  http://superleo.javaeye.com/rss


The first column are visited after a request, Rails process memory has increased by more than 9MB, the second column are processed the request, Rails process actual current accounts for more than 170 MB memory, the third column is the deal with what the request.

According to the statistical results, you can easily identify those that cause your Rails process memory leak culprit, and which you request to visit one of the Rails process memory on soaring thing is already clear at a glance, which is not very easy? In fact, through this approach, JavaEye took only half an hour, it solved the problem once more than half a year the memory leak problem, although the soil solution, very effective!
  • del.icio.us
  • StumbleUpon
  • Digg
  • TwitThis
  • Mixx
  • Technorati
  • Facebook
  • NewsVine
  • Reddit
  • Google
  • LinkedIn
  • YahooMyWeb

Related Posts of Rails to monitor the process of memory leaks skills

  • Hibernate II Study Notes

    11. Many-to-many Of many that can be converted to two one-to-many <set name="students" table="teacher_student"> <key column="techer_id"/> <many-to-many column="student_id"/> </set> many-to-many data only from one end of the mainten

  • The company first entered the company I would like to Optimize SQL project

    I was a time not long into the company of people, but also a person to be graduated, but immediately entered the project team, involved in the development. The project is to make a China Telecom Operation System, is a big system. My so-called develop ...

  • js page Jump implementation of a number of ways

    The first is: <script language="javascript" type="text/javascript"> window.location.href = "login.jsp? backurl =" + window.location.href; </ script> The second: <script language="javascript"> alert

  • Dynamic loading JS script four kinds of methods

    To achieve dynamic loading JS script has four kinds of methods: 1, direct document.write <script language="javascript"> document.write ( "<script src='test.js'> <\ / script>"); </ script> 2, dynamic scri

  • Hibernate primary key strategy-sequence

    Today, the use of hibernate in the company encountered a troublesome problem, the use of hibernate when the primary key generation strategy set sequence, but always reported in the implementation could not get next sequence value of the error, then o ...

  • Hibernate pessimistic locking mechanism for locking and optimistic locking

    hibernate lock mechanism 1. Pessimistic lock It refers to the modification of data by outsiders hold a conservative attitude. The assumption that at any time access to data, may also have another client to access the same data, in order to maintain t ...

  • Nan-Jing 5: When IBatis.Hibernate mixed affairs, pay attention to your SQL

    [Problem] Now, in the development of the so-called multi-storey JavaEE applications, data persistence layer is always essential, and "Automatic" of ORM - Hibernate, and "all-manual-type" of SqlMap - IBatis, equivalent data are Per ...

  • hibernate to use the principle of

    The use of hibernate, implementation of data persistence. Has the following several processes. One configuration database connection information. Hibernate.config 2 configuration mapping. 3 use: the use of the process are the following steps: 3.1: Ge ...

  • Based on JDBC, JPA Annotation achieve simple CRUD Generic Dao

    The origin of ideas are pretty long history of reasons: [Use iBATIS history] The use of iBATIS has been a long time, the system is to use the CRUD template tool to generate the code, although there are tools to generate, but looked at a lot of CRUD the Sq

  • Hibernate's lazy strategy

    hibernate Lazy strategy can be used in: <class> tag, it can be true / false Tags can <PROPERTY> values true / false type of necessary tools to enhance <set> <list> can tag values true / false / extra <many-to-one> <on ...

blog comments powered by Disqus
Recent
Recent Entries
Tag Cloud
Random Entries