Seeding mongodb via docker-compose

As a developer checking out a project or codebase, I liken the experience to going to a car dealer for a test drive. I just want to get to the “test drive” as quickly as possible – I don’t want to sign a bunch of papers, talk with multiple employees, and spend time having to go through an instructional safety training manual – I just want to drive. In the software development world, this means no configuring of complex options, no heavy setup of dependencies required, and no reading through complex documentation to figure out how to get the app running.

It’s possible that a stateless service, think web app and app server, will have a dependency on a stateful service, think database with existing values.  To enable a developer to test drive the web app locally, there are a couple options:

1. Point the local web app + app server to a shared database instance
2. Package a pre-seeded database to run locally as part of the developer setup

Option 1 is not preferable for scenarios where the shared database instance must either not be mucked with or is behind a security gate.

Option 2 necessitates running the database locally with pre-seeded data as well as pointing the dev app server code to it. Fortunately, docker-compose enables this experience in a really seamless way. The remainder of this short post will describe option 2 via docker-compose.

docker-compose-dev.yaml

mongodb:
  image: mongo
  ports:
    - "27017:27017"

mongo-seed:
  build: ./deploy/mongo-seed
  links:
    - mongodb

redis:
  image: redis
  container_name: cache
  expose:
    - 6379

web:
  restart: always
  image: node:8
  volumes:
      - ./:/usr/src/app
  working_dir: /usr/src/app
  ports:
    - "3001"
  env_file:
    - .env
  environment:
    # mongodb:27017 is a direct reference to the 'mongodb' mongo container
    - DB_CONN_STRING=mongodb://mongodb:27017/assyrian
    - REDIS_URL=redis://cache
    # REACT_PARSE_APP_ID=REACT_PARSE_APP_ID
    # REACT_PARSE_JS_KEY=REACT_PARSE_JS_KEY
    # REACT_PARSE_SERVER=REACT_PARSE_SERVER
  links:
    - mongodb
    - redis
  command: bash -c "npm install && npm run start"

lb:
  restart: always
  image: dockercloud/haproxy
  links:
    - web
  volumes:
    - /var/run/docker.sock:/var/run/docker.sock
  ports:
    - 80:80

deploy/mongo-seed/Dockerfile

FROM mongo

COPY archiveDictionaryDefinition.agz /archiveDictionaryDefinition.agz
COPY archiveDictionaryWordDefinitionList.agz /archiveDictionaryWordDefinitionList.agz
COPY archiveSearchStat.agz /archiveSearchStat.agz
COPY archiveProverb.agz /archiveProverb.agz

RUN sleep 5
CMD mongorestore --host mongodb --drop --gzip --archive=/archiveDictionaryDefinition.agz --db=assyrian --collection=DictionaryDefinition ; mongorestore --host mongodb --drop --gzip --archive=/archiveDictionaryWordDefinitionList.agz --db=assyrian --collection=DictionaryWordDefinitionList ; mongorestore --host mongodb --drop --gzip --archive=/archiveSearchStat.agz --db=assyrian --collection=SearchStat ; mongorestore --host mongodb --drop --gzip --archive=/archiveProverb.agz --db=assyrian --collection=Proverb

Note, the above commands to mongorestore are used instead of mongoimport because the application makes use of the mongodb text indexes. Importing json files does not retain indexes. The app won’t function without these indexes; hence, the mongorestore is used to create the collections.

To run the app locally, execute

docker-compose -f docker-compose-dev.yaml up --build

What happens?

1. Runs the ‘mongo’ image
2. Runs the ‘mongo-seed’ image and ‘web’ which are dependent on the ‘mongo’ image
3. Runs redis cache as a container
4. Runs web app pointing to mongo and redis
5. Runs haproxy loadbalancer directing traffic to web app

You now have a local development environment which includes a seeded database and the web app + app server.  Extra credit: We can enable the load balancer container ‘lb’ to load balance requests from localhost:80 to each of the web app containers.  Running the docker-compose command with –scale web=3 would start up 3 web instances that are fronted by the load balancer.

Try spinning up multiple web apps

docker-compose up --build --scale web=3

The github repo for this code can be found here: https://github.com/sogwiz/assyrian_dictionary_web

Hope you enjoyed the test drive, vroom vroom!

UPDATE: To allow hot-reloading with the dockerized environment (developer can make local changes without having to rebuild docker env), use the docker-compose-dev.yaml file instead. It is inspired via this blog post which goes into yet a further level of enhancement if you are so inclined.