Wednesday, May 25, 2011

Realizing Build Pipelines: The First Pipeline

So, for my little post series about How to realize a Continuous Delivery Pipeline with tool X? I am looking for an appropriate first example pipeline.

I am trying to start with a fairly easy, but nevertheless realistic kind of pipeline... The choice is influenced by some pipelines I saw working in production. Of course, it's simplified.

Pipeline #1. Simple pipeline w/ artifact and manual trigger
  • commit stage -- compiles the code, executes unit tests and produces a war artifact
  • deploy to staging -- deploys the war file to a staging test server
  • acceptance test -- executes the 
  • deploy to production -- deploys the war file to production
Ok, a fairly easy one. But how it should be configured in detail?

  • triggering -- The last step is to be started manually, all others are triggered automatically.
  • version control - The first build uses a version control checkout for the application code, the 2nd and 4th one a checkout from a deployment code repo and the tests are located in another repository. 
Of course, it's more a kind of anti-pattern to have production code and tests separated from each other. Anyway, often this is the case and I would like to have scenario which does not have two jobs which require to checkout the same version/revision of the source code.

Sunday, May 15, 2011

Drag-and-Drop with SVG with Rails Persistence (Part 1)

As a spike for my pipeline dashboard project I would like to have a page with some drawn rectangles which I can move around. Furthermore, the position of the moved objects should be persisted.


Since I like to get more hands on experience with Ajax websites which communicate with the server using JSON, the outline of my solution is the following:

  • The rectangles are drawn using SVG.
  • The rectangles are moved around using drag-and-drop.
  • There is a Rails application which provides a RESTful web service to retrieve the positions of the rectangles and of course, to persist changes of the positions. (This is inspired by an example in "RESTful web services" by Richardson and Ruby.)
  • To simplify the usage the example page is provided as the index page of the Rails web application.

Creating the RESTful service

Creating a RESTful web service with rails is very simple. You will see, it takes just a few minutes.

$ rails spike-dnd-with-svg-and-rails
      create  app/controllers
      create  app/helpers
      create  app/models
      create  app/views/layouts
      create  config/environments
      create  config/initializers
      create  config/locales
      create  db
      create  doc
      create  lib
      create  lib/tasks
      create  log
      create  public/images
      create  public/javascripts
      create  public/stylesheets
      create  script/performance
      create  test/fixtures
      create  test/functional
      create  test/integration
      create  test/performance
      create  test/unit
      create  vendor
      create  vendor/plugins
      create  tmp/sessions
      create  tmp/sockets
      create  tmp/cache
      create  tmp/pids
      create  Rakefile
      create  README
      create  app/controllers/application_controller.rb
      create  app/helpers/application_helper.rb
      create  config/database.yml
      create  config/routes.rb
      create  config/locales/en.yml
      create  db/seeds.rb
      create  config/initializers/backtrace_silencers.rb
      create  config/initializers/inflections.rb
      create  config/initializers/mime_types.rb
      create  config/initializers/new_rails_defaults.rb
      create  config/initializers/session_store.rb
      create  config/environment.rb
      create  config/boot.rb
      create  config/environments/production.rb
      create  config/environments/development.rb
      create  config/environments/test.rb
      create  script/about
      create  script/console
      create  script/dbconsole
      create  script/destroy
      create  script/generate
      create  script/runner
      create  script/server
      create  script/plugin
      create  script/performance/benchmarker
      create  script/performance/profiler
      create  test/test_helper.rb
      create  test/performance/browsing_test.rb
      create  public/404.html
      create  public/422.html
      create  public/500.html
      create  public/index.html
      create  public/favicon.ico
      create  public/robots.txt
      create  public/images/rails.png
      create  public/javascripts/prototype.js
      create  public/javascripts/effects.js
      create  public/javascripts/dragdrop.js
      create  public/javascripts/controls.js
      create  public/javascripts/application.js
      create  doc/README_FOR_APP
      create  log/server.log
      create  log/production.log
      create  log/development.log
      create  log/test.log
 $ cd spike-dnd-with-svg-and-rails

Now we use the scaffolding to create our domain object "item", which represents a rectangle to be drawn. An item has x and y coordinates, a name and an id.

$ ruby script/generate scaffold item id:integer name:text x:integer y:integer 
      exists  app/models/
      exists  app/controllers/
      exists  app/helpers/
      create  app/views/items
      exists  app/views/layouts/
      exists  test/functional/
      exists  test/unit/
      create  test/unit/helpers/
      exists  public/stylesheets/
      create  app/views/items/index.html.erb
      create  app/views/items/show.html.erb
      create  app/views/items/new.html.erb
      create  app/views/items/edit.html.erb
      create  app/views/layouts/items.html.erb
      create  public/stylesheets/scaffold.css
      create  app/controllers/items_controller.rb
      create  test/functional/items_controller_test.rb
      create  app/helpers/items_helper.rb
      create  test/unit/helpers/items_helper_test.rb
       route  map.resources :items
  dependency  model
      exists    app/models/
      exists    test/unit/
      exists    test/fixtures/
      create    app/models/item.rb
      create    test/unit/item_test.rb
      create    test/fixtures/items.yml
      create    db/migrate
      create    db/migrate/20110514212212_create_items.rb

Before starting the service we init the database as follows.

$ rake db:migrate
(in /Users/alex/SWDEV/spike-dnd-with-svg-and-rails)
==  CreateItems: migrating ====================================================
-- create_table(:items)
   -> 0.0014s
==  CreateItems: migrated (0.0015s) ===========================================

The service is started with a script contained in the folder script.

$ script/server --daemon
=> Booting Mongrel
=> Rails 2.3.5 application starting on

What we have so far?

We have a very simple web application running on http://localhost:3000 which allows for managing our domain objects. Its index page shows some welcome message with some hints. The interesting page is http://localhost:3000/items. It returns a list of our domain objects, which we called "item". Of course the list is empty.

To create our first entry we klick on "New item" and fill the form with id=1 and some fantasy. Now http://localhost:3000/items returns a list containing one item. Beside the HTML pages we can also request XML data:

$ curl -i http://localhost:3000/items.xml
TTP/1.1 200 OK
Connection: close
Date: Sun, 15 May 2011 14:00:00 GMT
ETag: "8fc46ac5da86d8127536a43e164a4878"
Content-Type: application/xml; charset=utf-8
X-Runtime: 7
Content-Length: 339
Cache-Control: private, max-age=0, must-revalidate

<?xml version="1.0" encoding="UTF-8"?>
<items type="array">
    <created-at type="datetime">2011-05-14T21:40:20Z</created-at>
    <id type="integer">1</id>
    <name>my first rect</name>
    <updated-at type="datetime">2011-05-15T10:39:21Z</updated-at>
    <x type="integer">3</x>
    <y type="integer">4</y>

Using the verbose option -i of curl tells us a lot more details about the request, especially its return code.

As the Rails application provides a RESTful service we can access the "item" with id 1 using
$ curl -i http://localhost:3000/items/1.xml
HTTP/1.1 200 OK
Connection: close
Date: Sun, 15 May 2011 14:01:24 GMT
ETag: "a388de3aec9d892bc1b1af2f81194701"
Content-Type: application/xml; charset=utf-8
X-Runtime: 7
Content-Length: 293
Cache-Control: private, max-age=0, must-revalidate

<?xml version="1.0" encoding="UTF-8"?>
  <created-at type="datetime">2011-05-14T21:40:20Z</created-at>
  <id type="integer">1</id>
  <name>my first rect</name>
  <updated-at type="datetime">2011-05-15T10:39:21Z</updated-at>
  <x type="integer">3</x>
  <y type="integer">4</y>

What about JSON?

Cool, out of the box we got HTML and XML. Let's try to use json. We either can tell cURL that we would like to get JSON results or we append the postfix ".json" to the URL,

$ curl -i http://localhost:3000/items.jsonl
HTTP/1.1 406 Not Acceptable
Connection: close
Date: Sun, 15 May 2011 14:10:41 GMT
Content-Type: text/html; charset=utf-8
X-Runtime: 5
Content-Length: 1
Cache-Control: no-cache

$ curl -i -H "Accept: application/json" http://localhost:3000/items
HTTP/1.1 406 Not Acceptable
Connection: close
Date: Sun, 15 May 2011 14:13:27 GMT
Content-Type: text/html; charset=utf-8
X-Runtime: 5
Content-Length: 1
Cache-Control: no-cache

Both calls are returning 406 and no content. So, there is something to do to enable JSON support.

How to ad JSON support?

To provide JSO we have to extend our controller ./app/controllers/items_controller.rb. The controller has code like this:
def index
    @items = Item.all

    respond_to do |format|
      format.html # index.html.erb
      format.xml  { render :xml => @items }

To add json support we simply copy each "format.xml" line and exchange "xml" with "json". That's it. (Of course, it would be very nice if an option for scaffolding could enable the json support.)

Let's verify whether the JSON support works.
$ curl http://localhost:3000/items/1.json
 {"item":{"name":"my first rect","created_at":"2011-05-14T21:40:20Z","x":3,"y":4,"updated_at":"2011-05-14T21:40:20Z","id":1}}


Cool. we have out first RESTful web service with JSON, XML (and also HTML) support...  written in less than 10 minutes. Not so bad.

With the next post I try to cover the manipulation of data using PUT and DELETE requests rather than GET requests.