| Project: | ProjectPier |
| Version: | 0.8.8-stable |
| Component: | Code |
| Category: | support request |
| Priority: | normal |
| Assigned: | Unassigned |
| Status: | patch - code needs work |
Description
I would like to see if we can possibly integrate some of this code into ProjectPier. Either as a separate script that generates a file to be fed into the installer, or possibly run directly within the installer script (perhaps both options).
The code has little documentation in it so feel free to ask
questions. Also, it's setup in a way that allowed it to run multiple
exports at once using a distributed network of machines on Amazon's
EC2 service - so a lot of the code is just for handling the EC2
stuff. The main interest would be in the convert/export.php file -
that's the meat of the export process.
| Attachment | Size |
|---|---|
| grid.zip | 107.9 KB |
Looks interesting.
You might also want to investigate the importer in RailsCollab, which instead of using the API reads from a dump file.
http://railscollab.rubyforge.org/svn/trunk/db/migrate_basecamp.rb
For our purposes, the file export/single_file.php seems to be actually better since most of the distributed network is not used.
However from the quick tests I have done, the includes and PHP modules have to be looked at better. By that I mean that I'm not sure this file was completely done.
Ryan suggested that instead of using this Basecamp Importer (more of an importer than an exporter), we break up the work in two parts:
* XML import/export of projects
* XML converter from a Basecamp dump to a ProjectPier dump which would then use the XML import/export module
Any suggestions on the definition of the XML format? Any XML expert around?
Found this today that might also be useful to look at. Its slightly dated though http://matthewhutchinson.net/articles/2006/12/18/basecamp-activecollab-importer
Some ideas:
It should be fairly straightforward to export XML that mirrors the SQL layout. The exporter could do a "mindless" dump of the db tables, with some intelligence to select subsets of tables for different kinds of exports, or project-limited exports.
A version code should be included in the XML output to help the importer identify conflicts or omissions.
The importer would need to ignore unrecognized fields. It should choke or warn on unknown versions.
---
A more intelligent design (as opposed to "mindless generic dump") may be called for, especially if we want to support exporting XML that is useful to other applications with different schemas or for use in generic XML handlers like Excel, IE, FF, etc.
---
A "full" export should be able to be used as a complete backup of the data (if not that of the "plugins").
---
The exporter and importer should both able to handle any custom plugin data. Of course the intelligence for this can be handed to the plugins via the Plugin API, when it exists.
Hello all,
Total newbie here. Just got our ProjectPier up and running and very excited to be end nearly 3 years of BaseCamp monthly fees.
I need to import our Basecamp XML file into our blank ProjectPier install. I found this post, and...I am not certain on where to upload the 'grid' folder and how to proceed. I understand there's very little documentation, perhaps I've missed it altogether.
Any input on the direction I need to take would be truly appreciated.
Thank you kindly in advanced!
Andrew
I just got my install up and ready. from the research i've done and in chatting with a few developers on this project it appears there is no import function available at this point - HOWEVER - there is a need and I know our company is considering investing the dollar to make the import happen as well as a few other features Basecamp doesn't offer like templating projects.
Anyways, if you are possibly interested in joining resources to get a developer to create the import function please let me know. As I said I am in process of getting proposal for this work.
For what its worth this software is licensed under the AGPL meaning we can do whatever we want to the program and even resell it. http://www.projectpier.org/files/agpl-3.0.txt
What software has been released under the AGPL? The current version of ProjectPier (0.8.0.3) has been relased under the HPL (Honest Public License). You are not allowed to sell projectpier and any code changes you make need to be freely submitted back to the project.
http://www.projectpier.org/about/honest-public-license-hpl
I can not speak to any future versions, some people talked about switching licenses with the upcoming release of 0.8.5.
-Jon
Hey Jon,
I guess it goes a little bit off-topic here, but I was looking at that this week and I'm not sure what license is used anymore. From http://projectpier.org/about:
"ProjectPier is freely available and licensed under the Gnu Affero General Public License (AGPL)"
"ProjectPier was started as a fork of the now commercial activeCollab application. This means older versions are available under the same license, the Honest Public License."
But the release and trunk version are distributed with the HPL. But Ryan recently told me on IRC that it was under the AGPL.
So I don't know which one is actually in effect right now...
From what I understand of the AGPL, it does mean that it's ok to sell the software (including things like hosting) as long as you make the source available as well. Modifications have to be made public (and under the same license) unless it's for a private use (hosting is not considered private use in the AGPL, which fill this exact loophole of the GPLv2)
I haven't looked at the HPL though, thinking PP switched to the AGPL.
Tim
Clearly 0.8.0.3, and previous versions (back to the initial relase, 0.8.0) were released and distributed with a license.txt file that includes the HPL. The switch to the AGPL was targeted for version 0.8.5 which has not yet been released. My understanding, based on an email I received from Ryan, was that his main reason for wanting to switch is because the HPL is not recognized by the OSI, whereas the AGPL is. I believe both licenses (HPL and AGPL) effectively add onto the GPL by closing the SAAS loop. Meaning that anyone providing a ProjectPier hosting service is required to make all of their code changes publicly available for download.
At first I was a little alarmed by the whole idea of someone "selling" copies of ProjectPier, but then I read this: http://www.gnu.org/philosophy/selling.html and realized it's actually encouraged.
I'm currently working on version 0.8.5 and would like to feel comfortable that this issue has been completely resolved prior to the first beta release.
Has there been any movement on this issues yet? I'm also keep to move my data away from basecamp too.
I have written a single PHP script that will convert a Basecamp XML file into SQL commands to import data. It is ugly, but it works.
Not all data in Basecamp matches with PP, but I tried to get as much as I can. Anything I couldn't map leaves the field as either the "DEFAULT", or they are omitted from the INSERT.
There are some configuration options at the top that should be filled in to match your PP install. There are a couple of lookup functions at the bottom of the file for IM and Timezone values. I didn't add many timezones in, it will default to GMT if a match is not made.
By default this is a non destructive script that will output user/password information plus a huge bunch of SQL. You can make it destructive my removing a few lines near the bottom.
The SQL should only be used against a CLEAN INSTALL of PP. DO NOT use this against a working copy because it will DESTROY your data. This is meant as a one-time-only conversion script.
Basic usage:
1. Generate a Basecamp XML archive.
2. Download PP 0.8.5 (may work with other versions)
3. Run the installation and test if you want
4. Configure script as required and run via command line or browser (PHP)
5. Review SQL or look for script errors
6. Enable execute in script, or run through mysql manually
7. Enjoy your Basecamp projects in PP.
Good luck.
(Make sure you rename the downloaded file to bc2pp.php, removing the underscores and txt extension)
tgf,
This script works quite well!
It doesn't add recent activities to Project Pier though.
Everything else works as expected.
If you could figure out how to get recent activities working from Basecamp, this would be the best free, Basecamp importer available.
Hi rayray,
I'm glad it worked for you.
There are a number of features of Basecamp that are not made available in the XML export. I did not see any information from Writeboards, Files, Project Permissions, Recent Activity or possibly others.
I am going to be manually exporting my Writeboards and add them into PP manually, possibly with the wiki patch.
I think the Basecamp API may provide access to more of these items, but I did not want to spend the time learning it just to migrate away from Basecamp. The original post has code using the API, but I found the XML export easier to work with.
(5 Minutes later)
Actually, thinking about recent activity, I may be able to generate this from the various date-time information in the xml file. If I get some free moments later this week I will try to investigate if the data can be mapped.
Thanks for the idea.
One other thing I noticed, tgf, is comments on messages have the wrong timestamp.
Not sure if it's possible to fix this, but thought I'd let you know.
[EDIT]
I'm using the task comments patch located here:
http://www.projectpier.org/node/1198
Another nice-to-have would be importing Basecamp's to-do comments into PP's task comments.
Not sure if this is doable, but thought I'd mention it!
Rayray, thanks for the testing. I guess I didn't quite pay enough attention to the message comments when I did my import. I have found that the issue has to do with the fact that the date/time displayed for the comments is actually from the "updated" date field in PP. I had only imported the "created" date field. I will make the change in my script.
In the mean time, the following SQL can be run to correct this issue without redoing the entire import:
UPDATE comments SET updated_on = created_on, updated_by_id = created_by_id WHERE updated_on = '0000-00-00 00:00:00' AND updated_by_id IS NULL
I have not installed or looked at the to-do comment patch, but when I do I will look to updating the import accordingly.
Thanks tgf.
That comments SQL query fixed the message timestamps.
Hope you can get to-do comments working as well!
FYI, I'll be releasing a CSS patch for the acSimple theme to make PP look a lot like Basecamp. So if any of you are using that theme, look out for that.
Has anyone used this script on a recent XML export of basecamp data?
Thought I made the mods required for it to work with 0.8.6-stable - no luck so far.. Doesn't want to work for me.
A little help would be most appreciated.
Thanks!
There really should be something current available to import basecamp data for project pier users.
If needed, perhaps I could hire someone to re-develop the above basecamp data import script as needed for project pier 0.8.6-stable and share it with the community...?
Thanks!
Hello;
is that someone has found a solution to import data basecmphq to version 0.8.8
thank you
Hi, I have tried with 0.8.8 and getting error:
SQL
BEGIN WORK;
DELETE FROM pp088_companies;
ALTER TABLE pp088_companies AUTO_INCREMENT = 1;
INSERT INTO pp088_companies (client_of_id, name, homepage, address, address2, city, state, zipcode, country, phone_number, fax_number, timezone, created_on, created_by_id, updated_on, updated_by_id) VALUES (0,'WebWave.com', '', '', '', '', '', '', '', '', '', -5, '2012-04-06 03:44:14', 1, '2012-04-06 03:44:14', 0);
INSERT INTO pp088_companies (client_of_id, name, homepage, address, address2, city, state, zipcode, country, phone_number, fax_number, timezone, created_on, created_by_id, updated_on, updated_by_id) VALUES (1,'Alex Ikenson', '', '', '', '', '', '', '', '', '', -6, '2012-04-06 03:44:14', 1, '2012-04-06 03:44:14', 1);
INSERT INTO pp088_companies (client_of_id, name, homepage, address, address2, city, state, zipcode, country, phone_number, fax_number, timezone, created_on, created_by_id, updated_on, updated_by_id) VALUES (1,'Clevaland Technologies', '', '', '', '', '', '', 'India', '', '', -6, '2012-04-06 03:44:14', 1, '2012-04-06 03:44:14', 1);
INSERT INTO pp088_companies (client_of_id, name, homepage, address, address2, city, state, zipcode, country, phone_number, fax_number, timezone, created_on, created_by_id, updated_on, updated_by_id) VALUES (1,'Colliers International', '', '', '', '', '', '', '', '', '', -5, '2012-04-06 03:44:14', 1, '2012-04-06 03:44:14', 1);
INSERT INTO pp088_companies (client_of_id, name, homepage, address, address2, city, state, zipcode, country, phone_number, fax_number, timezone, created_on, created_by_id, updated_on, updated_by_id) VALUES (1,'Degi', '', '', '', '', '', '', '', '', '', -6, '2012-04-06 03:44:14', 1, '2012-04-06 03:44:14', 1);
INSERT INTO pp088_companies (client_of_id, name, homepage, address, address2, city, state, zipcode, country, phone_number, fax_number, timezone, created_on, created_by_id, updated_on, updated_by_id) VALUES (1,'DNN Direct', '', '', '', '', '', '', '', '', '', -6, '2012-04-06 03:44:14', 1, '2012-04-06 03:44:14', 1);
INSERT INTO pp088_companies (client_of_id, name, homepage, address, address2, city, state, zipcode, country, phone_number, fax_number, timezone, created_on, created_by_id, updated_on, updated_by_id) VALUES (1,'People Serve', 'www.pserveasia.com/', '16th Floor, Maneeya Center Building', '518/5 Ploenchit Road, Lumpini, Pathumwan', 'Bangkok', '', '10330', 'Thailand', '', '', -5, '2012-04-06 03:44:14', 1, '2012-04-06 03:44:14', 1);
INSERT INTO pp088_companies (client_of_id, name, homepage, address, address2, city, state, zipcode, country, phone_number, fax_number, timezone, created_on, created_by_id, updated_on, updated_by_id) VALUES (1,'Phuket Project', '', '', '', '', '', '', '', '', '', -5, '2012-04-06 03:44:14', 1, '2012-04-06 03:44:14', 1);
DELETE FROM pp088_users;
ALTER TABLE pp088_users AUTO_INCREMENT = 1;
INSERT INTO pp088_users (company_id, username, email, token, salt, twister, display_name, title, office_number, fax_number, mobile_number, home_number, timezone, created_on, created_by_id, updated_on, last_activity, is_admin, auto_assign) VALUES (1, 'rfurdzik@gmail.com', 'rfurdzik@gmail.com', 'ded1fdd6da85c82a60088d61b0508f868b048ff3', 'ddeab605e4823', '', 'Rafal Furdzik', 'CEO', '+1 718 717 2857 x', '', '+66 848 78 3087', '', -5, '2010-04-07 00:08:49', 0, '2011-10-28 06:37:21', '2012-04-06 03:44:14', false, );
INSERT INTO pp088_users (company_id, username, email, token, salt, twister, display_name, title, office_number, fax_number, mobile_number, home_number, timezone, created_on, created_by_id, updated_on, last_activity, is_admin, auto_assign) VALUES (1, 'rfurdzik@gmail.com', 'rfurdzik@gmail.com', 'ded1fdd6da85c82a60088d61b0508f868b048ff3', 'ddeab605e4823', '', 'Rafal Furdzik', 'CEO', '+1 718 717 2857 x', '', '+66 848 78 3087', '', -5, '2010-04-07 00:08:49', 0, '2011-10-28 06:37:21', '2012-04-06 03:44:14', false, );
Could not execute: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')' at line 1Rolled back
Hello
thank you for your script I managed to import the database
despite that there were problems with projects containing accenet
but the problem that I not display the message list in the project
is that you have a solution to this problem
thank you
I made the necessary changes to the author's original script to make it work with .8.8. Should import fine now. Some db fields were missing and not working correctly. Please note this is a destructive version...meaning run it only on a NEW install of Project Pier to import your Basecamp items.
Let me know if you have any problems.