| Project: | ProjectPier |
| Version: | 0.8.5.0-Beta1 |
| Component: | Code |
| Category: | bug report |
| Priority: | critical |
| Assigned: | phpfreak |
| Status: | closed - fixed |
Description
It seems ProjectPier needs to allocate enough memory to have a file it is going to send resident in memory... is this necessary?
I fixed this by streaming the file instead of having to have it resident in memory... quick hacky patch included.
The issue exists and the patch is a great solution. However, the patch only works for file based repositories. So, if the installation uses filesystem file repository we can use it. For installation using database file repository,
a) we can ignore the patch and suffer the memory issue
b) we can put files < 1 MB (not configurable) in database and rest on file and still use patch (hybrid solution)
c) after upload we split up a file in chunks of 1MB (not configurable) and put the chunks in the database and during download we put out the chunks in sequence
Putting large files in database and then putting them in temp file for download does not work. We still have to get the complete file from database before writing it to file and so still get memory issues.
For database, option c looks like the way to go.
Any comments / suggestions / solutions anyone?
Fixed in 0.8.6 for filesystem file repository only. Database file repository needs redesign. But it works great :-)
Agreed... 0.8.8 has the same issue... splits the uploaded file into chunks but if you're on a low ram cloud box like we are, you can't download large files... looks like anything chunked into rows craps out at the first 500k returned from the DB.
We kept getting download issues after about 500k or so, and the file is always corrupted.
We're going with filesystem for now which seems to work for everyone else.