Migrating from xp-dev.com to code.google.com

We have decided to start the migration from xp-dev.com to code.google.com (Google Project Hosting – GPH) . First, we have downloaded the source code from Subversion and uploaded it to a Mercurial repo in GPH.  Piece of cake! 😉

Then, the tough one;  we have had to migrate the issues from xp-dev to GPH but there’s NO API in xp-dev.com, so this weekend I have rolled up my sleeves and crafted a tiny PHP script for web-scrapping the bugs/issues list from xp-dev.com (you have to replace the USERNAME, PASSWORD and BUGS_ID with your appropriate fields):

<?
   //CURL stuff
   //This executes the login procedure
   $ch = curl_init();
   curl_setopt($ch, CURLOPT_URL, 'http://www.xp-dev.com/login/wicket:interface/:63:loginPanel:form::IFormSubmitListener::');
   curl_setopt($ch, CURLOPT_POSTFIELDS, 'username=USERNAME&password=PASSWORD');
   curl_setopt($ch, CURLOPT_POST, 1);
   curl_setopt($ch, CURLOPT_HEADER, 0);
   //curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
   curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
   curl_setopt($ch, CURLOPT_COOKIEJAR, "cookies");
   curl_setopt($ch, CURLOPT_COOKIEFILE, "cookies");
   curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
   //make sure you put a popular web browser here (signature for your web browser can be retrieved with 'echo $_SERVER['HTTP_USER_AGENT'];'
   curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12");
 
   curl_exec($ch);
 
   curl_setopt($ch, CURLOPT_URL, 'http://www.xp-dev.com/tracking/BUGS_ID');
 
   $subject = curl_exec($ch);
 
   foreach(preg_split("/(\r?\n)/", $subject) as $line){
       preg_match('/bugTitle">(.*)<\/h5><p>(.*)<\/p>/ism', $line, $trozos);
       if (count($trozos) > 0) echo html_entity_decode($trozos[1], ENT_QUOTES, 'UTF-8'). "~" . html_entity_decode($trozos[2], ENT_QUOTES, 'UTF-8') ."~";
 
   }
 
   curl_close($ch);
?>

So far, so good, but what are we supposed to do with that script? Well, you have to execute it from the command line and redirect the output to a file. For example:

$ php -q issue_scrapping.php  >  issues.txt

Now, the hardest part: you have to import that issues in GPH using the GData API for Issue Tracking. I’ve used the Java binding of the API. The only thing that I’ve had to customize in this example from the API is another little piece of Java code in the run() method:

               Scanner file = new Scanner(new File("issues.txt"));
		file.useDelimiter("~");
		while (file.hasNext()) {
			String summary = file.next();
			String description = file.next();
			// Create an issue
			IssuesEntry issueInserted = client.insertIssue(makeNewIssue(
					summary, description));
 
			String issueId = client.getIssueId(issueInserted.getId());
			System.out.println("Issue #" + issueId + " created");
 
		}
 
		file.close();

(and in the makeNewIssue() method so that it accepts the two new String parameters, summary and description). And here you have the result of this hand-made porting 😉

One thought on “Migrating from xp-dev.com to code.google.com

Leave a Reply

Your email address will not be published. Required fields are marked *