giampaolo · GitHub

From crusade...@gmail.com on February 13, 2014 14:31:57

I wrote a script that prints in real time the top 10 processes, sorted by CPU
usage (just like top).
# init CPU meters
for proc in psutil.process_iter():
   proc.get_cpu_percent(interval=None)
for i in range(100):
    time.sleep(1)
    for proc in psutil.process_iter():
        cpu = proc.get_cpu_percent(interval=None)
    ...
The above hogs a whopping 29% CPU. This is what cProfile says:
         2525223 function calls in 129.009 CPU seconds
   Ordered by: cumulative time
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      100  100.002    1.000  100.002    1.000 {time.sleep}
    59798    0.714    0.000   20.810    0.000
build/bdist.linux-x86_64/egg/psutil/__init__.py:679(get_cpu_percent)
    59798    0.158    0.000   16.206    0.000
build/bdist.linux-x86_64/egg/psutil/__init__.py:1166(cpu_times)
    59798    1.293    0.000   16.048    0.000
build/bdist.linux-x86_64/egg/psutil/_pslinux.py:270(get_system_cpu_times)
   119596   13.827    0.000   13.827    0.000 {method 'readline' of 'file' objects}
   194235    0.487    0.000   11.195    0.000
build/bdist.linux-x86_64/egg/psutil/_pslinux.py:460(wrapper)
    59898    0.197    0.000    4.329    0.000
build/bdist.linux-x86_64/egg/psutil/__init__.py:1031(process_iter)
    59795    0.139    0.000    3.815    0.000
build/bdist.linux-x86_64/egg/psutil/__init__.py:830(is_running)
    59798    0.805    0.000    3.566    0.000
build/bdist.linux-x86_64/egg/psutil/_pslinux.py:576(get_cpu_times)
    59802    0.717    0.000    3.494    0.000
build/bdist.linux-x86_64/egg/psutil/_pslinux.py:597(get_process_create_time)
    59798    0.091    0.000    3.359    0.000
build/bdist.linux-x86_64/egg/psutil/__init__.py:760(get_ext_memory_info)
    59798    1.210    0.000    3.157    0.000
build/bdist.linux-x86_64/egg/psutil/_pslinux.py:626(get_ext_memory_info)
   254133    2.902    0.000    2.902    0.000 {open}
   134433    2.726    0.000    2.726    0.000 {method 'read' of 'file' objects}
   254129    1.601    0.000    1.601    0.000 {method 'close' of 'file' objects}
   254329    0.697    0.000    0.697    0.000 {method 'split' of 'str' objects}
The problem is that every single call to get_cpu_percent() internally invokes
sum(cpu_times()). With maybe some weird exceptions (hybernation?) that should
be pretty much the same as calling time.time() and multiply by the number of
CPUs - which is MUCH faster.
Same benchmark, replacing get_cpu_percent() with get_cpu_times(), I get a more
reasonable 8% CPU usage:
         1323713 function calls in 108.063 CPU seconds
   Ordered by: cumulative time
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      100  100.000    1.000  100.000    1.000 {time.sleep}
   119751    0.328    0.000    7.363    0.000
build/bdist.linux-x86_64/egg/psutil/_pslinux.py:460(wrapper)
    59975    0.165    0.000    4.350    0.000
build/bdist.linux-x86_64/egg/psutil/__init__.py:1031(process_iter)
    59869    0.140    0.000    3.884    0.000
build/bdist.linux-x86_64/egg/psutil/__init__.py:830(is_running)
    59875    0.094    0.000    3.712    0.000
build/bdist.linux-x86_64/egg/psutil/__init__.py:743(get_cpu_times)
    59876    0.680    0.000    3.579    0.000
build/bdist.linux-x86_64/egg/psutil/_pslinux.py:597(get_process_create_time)
    59875    0.788    0.000    3.457    0.000
build/bdist.linux-x86_64/egg/psutil/_pslinux.py:576(get_cpu_times)
   119750    2.677    0.000    2.677    0.000 {method 'read' of 'file' objects}
   119751    1.340    0.000    1.340    0.000 {open}
   119750    0.546    0.000    0.546    0.000 {method 'close' of 'file' objects}
   119750    0.440    0.000    0.440    0.000 {method 'split' of 'str' objects}
    59875    0.297    0.000    0.297    0.000 {method 'rfind' of 'str' objects}
      100    0.000    0.000    0.235    0.002
build/bdist.linux-x86_64/egg/psutil/__init__.py:1016(get_pid_list)
      100    0.140    0.001    0.234    0.002
build/bdist.linux-x86_64/egg/psutil/_pslinux.py:359(get_pid_list)
I'm sure it's possible to improve even further, as top only uses 1% - not
sure how tough.

Original issue: http://code.google.com/p/psutil/issues/detail?id=477

Read the original on github.com ↗