Getting FlockData running leveraging the on-demand of things and the like
Having had great success with Bamboo over the last few weeks a simple cost benefit analysis showed hosted model was the best way to go for our needs. We seem to be “spinning” lots of things up of late. We’re currently running things through AWS instances as they are nicely integrated in to Bamboo. Still there are quite a few concepts to understand, but it’s all good as there are also ideas we can add to our own design- no point re-inventing the wheel.
So now we all code building with Bamboo running 110 tests on an AWS deployment fully integrated of our hosted Jira. It’s all very efficient and the support has been excellent. The trickiest thing to understand is really the AWS pricing model. The free micro instance isn’t sufficient enough to run Neo4j, RabbitMQ, 2x Tomcat, ElasticSearch and REDIS + code which is not unreasonable. So when we build up a bunch of stuff to test an instance is turned on in AWS and everything get’s built with the results being managed in the hosted Bamboo. Next thing will be to look at the long term costs of remote agents and decide on the deployment strategy. Bamboo may prove a little too expensive for building bigger clusters.
Our strategy now gives the the room to add performance and scalability instances together and observe behaviour on different hardware for the price of turning on the right machine configuration. So a fair bit of effort to get it going, but not unreasonable and mostly part time. It’s a credit to the community of collaborative development how well and functional integration across applications is getting. This is exactly what we’re doing with FlockData – making it painless to build a Business Graph and Search Engine.
Really glad that my ISP this week has now removed all data caps as I seem to be moving more data that ever around the place. Do the down/uploads of things grow to fill the available bandwidth?
Functionality wise we got a chord diagram up and going. This is off of some medical data we’re looking at collaborations between contributors on papers. The thicker the line the more often the participants have collaborated. Nice to see some pictures surfacing. Thanks to Sherry and Sathish for this.
Speaking of documentation, it also got quite a kick this week. Still a whiles to go but the API stuff should be a heck of a lot easier to figure out. There is also an Installation page that gives you the 5 commands necessary to get this going on your own machine. Having played around with AWS I can see us having an instance of this available for evaluation soon. That should make things ultra easy for everyone.