The CI Chain Reaction That Builds Flickr Downloadr
Update (6/11/2015) - Due to the recent news about malware on installers downloaded from SourceForge, I have decided to stay completely away from it. This is the reason for the move to GitHub releases for archiving the releases.
As written on the last entry in this blog, I finally got to write something on points 1) and 2) there - the CI (continuous integration) builds that are running in parallel on AppVeyor, Travis CI and Wercker and the multi-platform installers they create and publish to the website. I wrote this up as documentation on the main GitHub repository but decided to cross-post here on the blog too.
Automated Build Process
Every commit to the
master branch on the source code repository kicks off CI builds (that builds and runs all the unit tests) in three different CI services:
Any commit that has the string
[deploy] in the commit message will also build the installers for all three supported platforms, using BitRock InstallBuilder.
The installers will be committed separately from the CI builds to a new branch named
deploy-v<VERSION> (as an example, for the version
v188.8.131.52, the new branch will be
deploy-v184.108.40.206) on the
A custom webhook on the
flickr-downloadr/flickr-downloadr.github.io repo that runs on Heroku ensures that installers for all three platforms have been built successfully and then makes a commit with the name of the new branch, updated into the
branch file on the
flickr-downloadr/releases repository (
deploy-v220.127.116.11, in the example above) and sends an email a deployment will be ready to be pushed to the website soon [see here].
In the event that any of the CI builds fail to create the installer, after at least one installer gets successfully built and committed, the custom webhook waits for 30 minutes from the first installer getting committed and sends an email notifying of a build failure [see here].
Yet another webhook on the
flickr-downloadr/releases repo kicks off another build on Wercker to merge the new branch with all three installers (
deploy-v18.104.22.168 branch) into the
source branch on
flickr-downloadr/flickr-downloadr.github.io to make it ready for push into the
master branch that runs the main website. This CI job does a few other things like archiving this version to
SourceForgeGitHub releases etc., as can be seen here.
The final push to the website is manual and can be done by running
grunt deploy on the latest, merged version of the
source branch and this will make the latest version installers current on the website.
Batch Download Flickr Photos From Windows, Mac or Linux!
The previous entry on this blog discussed the idea of probably creating a native, Cocoa, Mac OS X port for the then, Windows-only, WPF version of the app.
Any action started much later from the date of the post; to be precise, more than two months later with the creation of a new repository with this commit. The research that happened in-between helped decide on a third alternative - that of completely porting over the .NET C#/WPF into a Mono, C#/GTK# app that could work on all of Windows, Mac and Linux.
And exactly after two months and one day the v1.0 of the new GTKSharp enabled flickr downloadr was published on this website.
It really was an exhilarating journey of many frustrations, joy, learnings and moments of bliss - and there are a few things that I would like to follow up in separate entries here. Like:
- the parallel, multi-platform continuous integration/deployment that happens on AppVeyor (Windows), Travis CI (Mac) and Wercker (Linux).
- the building of a cross-platform installer using the free-for-open-source Enterprise edition of Install Builder provided generously by the awesome guys at BitRock.
- the slight face-lift applied on the website along with the cut-over to a new, Yeoman scaffolded web-app - complete with its automated
gruntbuild for minification, image compression etc. (do a quick View Source on this page to see the results)
…and many other such learnings and experiences.
So, please spread the word of this new, free, open-source utility for batch downloading photos from flickr that is available on Windows, Mac OS X and Linux (Ubuntu & Mint have been tested).
Contemplating a Mac Port of Flickr-downloadr
It’s been a long time since something has been posted on this blog! And really, it’s been quite a while since something has been done with the app at all.
And the main reason for that is that I have been focussing more on mobile apps and web apps lately. Combine this with the fact that a Macbook Pro is the primary machine these days and there you have the reason for why this Windows/Visual Studio application has not been getting any attention at all.
But because there is the Mac and there is an increasing desire to learn Cocoa and Objective-C, I have been toying with the idea of porting over the current WPF-based Windows desktop application into a Mac app.
Stay tuned for updates!
Single-click Deployment of WPF Application to GitHub Pages
This post will try to detail:
- how the NAntContrib Version task is automating the version increment; and then building the flickr downloadr WPF app
- how the app is being published (via NAnt and MSBuild), for deployment to the custom-installation url, which is being hosted at GitHub Pages)
- how some shell scripting helps to automate the upload of this published application to the site
The downloads page on flickrdownloadr.com has a link to the latest published version of the application listed there along with the current version number. For the first few times, this page was being updated manually. But because it’s still very early phases of development, where we are going to be releasing quite a few times and also because release early, release often is a good philosophy to try and practice, the need to automate this app-deployment-process soon arose.
Ideally, the publish process detailed here would be run from a CI environment, but that is not happening right now for a couple of reasons:
- There are no free and hosted CI services that runs the .NET version (v4.5) that flickrdownloadr is built on. Even though Codebetter CI has kindly agreed to use their service for this project and there were quite a few successful builds earlier, they do not support v4.5 yet.
- The deploy shell script (that is explained below) relies on the SSH identity to authenticate to GitHub to be already established when it runs. I haven’t figured out how else to handle the GitHub authentication (with write access) from a CI environment, without publicly storing the credentials in the repository.
The app is being published using the ClickOnce deployment methodology that is built-in to the Visual Studio IDE. Publishing a WPF application using ClickOnce manually, from within the Visual Studio IDE is a pretty straight-forward process and there are good tutorials to guide through the step-by-step wizard. But publishing from within the IDE is hardly something we could run as part of an automated setup.
Hello, and a hearty welcome to the flickr downloadr blog!
flickr downloadr was conceived back in May 2012 when I surprisingly discovered that even with the awesome API that flickr provides there are no free and/or open source app that lets one download their photos along with the metadata. I also wanted to learn the WPF and flickr downloadr was born.
But soon after the authentication modules using OAuth were done, the enthusiasm died and there was a brief hiatus in the development and it is recently that it has gained some momentum again. As of this writing, the latest beta version of the app works on any version of Windows that can run .NET framework 4.5, which is Windows Vista (untested), Windows 7 and Windows 8.