Support

UNiTE, Remote CLI, eXtract Wizard

#24253 UNiTE - Out of memory when doing Wpreplacedata

Posted in ‘UNiTE and Remote CLI’
This is a public ticket

Everybody will be able to see its contents. Do not include usernames, passwords or any other sensitive information.

Environment Information

PHP version
n/a
Tool
UNiTE
Tool version
n/a

Latest post by on Sunday, 28 February 2016 17:20 CST

perfect-web
 I have a big WordPress site which backups has size of 4,5GB. Mostly because of images. UNiTE does not have a problem with making and downloading backup in chunks. But when it starts the step Wpreplacedata the memory usage drastically increase. Database SQL files have size of 180 MB, so there is many data to precess.

To deal with it I have edited the file unite/engine/scripting/wordpress.ini
and removed ,wpreplacedata as I do not need it.

Maybe you could consider to make some changes in the method UStepWpreplacedata::_run.
Here are some of my ideas.

Run select query with some limit
$query = $db->getQuery(true)->select('*')->from($db->qn($tableName))->limit($offset, 100);
$data  = $db->setQuery($query)->loadAssocList();

and create while loop enclosing it and
if ( !empty($data)) { ... }


Also you could change
foreach ($data as $row)

to
foreach ($data as $i => $row)

and at the end of loop remove this item
unset($data[$i]);


I do not know exactly which variable causes memory leak, but maybe unsetting all variables at the end of loop cycle, used inside loop, would help.

nicholas
Akeeba Staff
Manager
Hello Piotr,

You are right. This code was copied from ANGIE that only does replacements a batch at a time. I will get back to you with a dev release.

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

nicholas
Akeeba Staff
Manager
I have just published a new dev release at https://www.akeebabackup.com/download/developer-releases/unite-dev.html Can you give a try, please?

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

perfect-web
Thank you. I will try to test it on Monday and I will get back to you

perfect-web
Hello,
unfortunately I have no more access to the website which was causing this bug. The client is not responding. I do not have another similar case to test it. If I will get a respond from my client I will try to test it.

I have tested it only on other smaller website and it is working.

By the way, we have made on enhancement in the code to retry download backup file if it has failed. We have split backup in Akeeba Solo into parts. And when we are downloading big backups, there are often connections problem, and we had to start UNiTE once again. Now it is just retrying on its own and everything is working correct.

If you are interested I can send you those changes. Just tell me where I can put it.

nicholas
Akeeba Staff
Manager
Yes, I'd like that. Can you give me your GitHub username so I can add you to our repository as a contributor, allowing you to make a PR?

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

perfect-web
My username on GitHub is: piotrmocko

nicholas
Akeeba Staff
Manager
Thanks! I added you as a read-only collaborator. It allows you to clone the repository so you can send us your pull request :)

Nicholas K. Dionysopoulos

Lead Developer and Director

🇬🇷Greek: native 🇬🇧English: excellent 🇫🇷French: basic • 🕐 My time zone is Europe / Athens
Please keep in mind my timezone and cultural differences when reading my replies. Thank you!

System Task
system
This ticket has been automatically closed. All tickets which have been inactive for a long time are automatically closed. If you believe that this ticket was closed in error, please contact us.

Support Information

Working hours: We are open Monday to Friday, 9am to 7pm Cyprus timezone (EET / EEST). Support is provided by the same developers writing the software, all of which live in Europe. You can still file tickets outside of our working hours, but we cannot respond to them until we're back at the office.

Support policy: We would like to kindly inform you that when using our support you have already agreed to the Support Policy which is part of our Terms of Service. Thank you for your understanding and for helping us help you!