{"id":369,"date":"2010-08-28T19:01:59","date_gmt":"2010-08-28T18:01:59","guid":{"rendered":"http:\/\/kudzia.eu\/b\/?p=369"},"modified":"2013-05-03T12:57:26","modified_gmt":"2013-05-03T11:57:26","slug":"php-adodb-memory-usage","status":"publish","type":"post","link":"https:\/\/kudzia.eu\/b\/2010\/08\/php-adodb-memory-usage\/","title":{"rendered":"php \/ adodb &#8211; memory usage"},"content":{"rendered":"<p>i&#8217;ve got seemingly simple php script running every night on one of my servers&#8230; on the same machine i started to experience nightly oom-killer.<br \/>\ni&#8217;ve found out that adodb&#8217;s Execute statement can do something completely else than you would expect from it &#8211; it loads whole result into memory rather than going row-by-row.<br \/>\n<!--more--><br \/>\nsimplified code:<\/p>\n<pre class=\"brush: php; title: ; notranslate\" title=\"\">\r\n$r = $c-&gt;Execute('SELECT * FROM big_table');\r\nwhile(($row=$r-&gt;FetchRow())!=false){\r\n  \/\/ process the data\r\n}\r\n<\/pre>\n<p>it was processing few gigabytes of results and recently &#8211; started to use more memory than was available on the server. problem was caused by <a href=\"http:\/\/phplens.com\/adodb\/reference.varibles.adodb_countrecs.html\">this feature of adodb<\/a>. apparently for mysqli driver it emulates row-counting by reading whole result to memory. magic fix &#8211; include this before heavy queries:<\/p>\n<pre class=\"brush: php; title: ; notranslate\" title=\"\">\r\n$ADODB_COUNTRECS=false;\r\n<\/pre>\n<p>and peak memory usage of my script dropped from 3GB dropped to 150MB.<\/p>\n<p>edit: even with <i>$ADODB_COUNTRECS=false;<\/i> i&#8217;ve still experienced memory use problems with <i>mysql<\/i> driver, once i switched to <i>mysqli<\/i> all worked fine.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>i&#8217;ve got seemingly simple php script running every night on one of my servers&#8230; on the same machine i started to experience nightly oom-killer. i&#8217;ve found out that adodb&#8217;s Execute statement can do something completely else than you would expect from it &#8211; it loads whole result into memory rather than going row-by-row.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17],"tags":[],"class_list":["post-369","post","type-post","status-publish","format-standard","hentry","category-tech"],"_links":{"self":[{"href":"https:\/\/kudzia.eu\/b\/wp-json\/wp\/v2\/posts\/369","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/kudzia.eu\/b\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/kudzia.eu\/b\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/kudzia.eu\/b\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/kudzia.eu\/b\/wp-json\/wp\/v2\/comments?post=369"}],"version-history":[{"count":4,"href":"https:\/\/kudzia.eu\/b\/wp-json\/wp\/v2\/posts\/369\/revisions"}],"predecessor-version":[{"id":1766,"href":"https:\/\/kudzia.eu\/b\/wp-json\/wp\/v2\/posts\/369\/revisions\/1766"}],"wp:attachment":[{"href":"https:\/\/kudzia.eu\/b\/wp-json\/wp\/v2\/media?parent=369"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/kudzia.eu\/b\/wp-json\/wp\/v2\/categories?post=369"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/kudzia.eu\/b\/wp-json\/wp\/v2\/tags?post=369"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}