When we are doing unit testing, more often than not we will probably stumble upon external dependencies that we wish we could just get them out of the way to focus on the testing task at hand. That is when mocking, stubbing, faking comes in handy.
If we apply these techniques in a broader context, says for app testing, where our external dependencies are web services, how do we consistently and reliably test our app against the ever changing data returned by real API calls? That is when we need a sandbox server.
The term sandbox is commonly used for the development of Web services to refer to a mirrored production environment for use by external developers. Typically, a third-party developer will develop and create an application that will use a web service from the sandbox, which is used to allow a third-party team to validate their code before migrating it to the production environment.
Now, when we talk about sandbox here, I mean our own, self-developed, local sandbox server, not an external sandbox server as provided by our service provider (e.g. Paypal, eBay, Google sandbox environment). Why? Because if we use an external sandbox server, it is still an external server, populated with test data from other users. We are often forced to use one so that we do not polute the service provider’s production data, at the cost of real network latency (if not worse).
Eat your own dog food
The fact that we depend on someone else’s service does not mean that we have to depend on it during development as well, at least for sections of app that consume data versus those that produce data. Well, we may still need it at some point before migrating to production; but for day-to-day development, one can totally make do without it.
We start by mirroring the real service, by constructing a fake one, producing just enough information that our app needs to consume to be functional (not essentially in a logically correct way).
/v0/topstories.json- to retrieve list of top stories
/v0/item/<id>.json- to retrieve a specific story with
I like Ruby, it makes rapid prototyping easy; and sinatra seems to be the perfect pick to quickly construct a dumb sandbox server. But any web frameworks in any languages should be capable of doing the same thing here.
Done and dusted! Now every GET request to
/v0/topstories.json will give us 3 story IDs, and every
/v0/item/<id>.json will give us the same ‘Hello World’ story.
Let’s make it a little bit smarter in the next prototyping iteration.
Now whenever the sandbox server receives a request, it will return a JSON response as saved under the
TipKeep the sandbox responses well-organized. They can serve as handy examples / quick documentations of what we can expect from external services.
What if we want to simulate different scenarios, e.g. when the server encounters high traffic load and fails to return proper response?
Now when we want to test scenario ‘empty’, send a request to
/_sandbox/empty, the subsequent requests will look for response under ‘empty’ dataset:
To revert to the normal ‘default’ dataset, send a request to
This can keep going until we have a sandbox server that balances our needs and its simplicity. Check out the complete Hacker News API sandbox example at hidroh/hackernews-sandbox.
NoteIt is very important to keep simplicity in mind, as we do not want to end up building comprehensive web services that is production-ready! The dumber the sandbox server is, the easier it is to maintain/update it.
###Switching to local sandbox server
With our sandbox server ready to be used for development and testing, we need to find a way to build our app against different servers: a sandboxed one for
debug build, and a real one for
release build. Here http://localhost:4567 is the default sinatra host.
Or even better, get a Raspberry Pi and use it to power a web server!
###Conclusion A local sandbox server makes it very quick and convenient to develop/test/demo our app.
- It saves us from being blocked from development on the dark days when the internet connection is slow, or when the real server runs into troubles.
- It facilitates pipelining of full-stack development, allowing app development with the assumption that the ‘to-be-developed’ web server will function in certain ways, without having to wait for it to be ready first.
However, keep in mind that it is just a mirror of the real thing. When the real thing changes, if we have no way to get ourselves notified of the changes, we would still be working against an outdated mirror of it!
Use an API validator, e.g. cucumber-api, and have it run against both the real and sandbox servers frequently, to ensure that we get notified as soon as they are not in sync.
We also need to constantly remind ourselves that our local sandbox server is a dumb one. It only produces fake responses for development purpose, not executing real server logic with sandbox data like those provided by Paypal or eBay. The decision to make it smarter or dumber is in our hands, and it is a trade-off between how much we want to mirror the real thing vs. how quickly we want to get it out of the way.
Now say good bye to the real web server, and start using a local sandbox server for a boosted development experience!