As part of the revamp of the DCO engine, we have been adding support for quite a few highly customizable scenarios for dynamic creatives - including storyboarding, geo-location, weather etc. If you haven’t seen them yet then you should give them a dekho - story-board demo, e-commerce demo and geo-loaction demo.
Considering we allow highly customized dynamic creatives, one of the challenges we faced was the trade-off between making our algorithms generic enough vs. flexibility to accommodate advertiser use cases.
Let’s a take an example for an Advertiser ‘A1’ who wants to show different creatives based on the current weather - sunny, cloudy, snow or rain. Another advertiser ‘A2’ who would want to show different creatives based on the current temperature - cold, pleasant or hot. Here is how these algorithms would look like in pseudo-code.
Even if we look at these two really simple use cases - the decision making for both has no commonality and in most systems would end up becoming two separate algorithms. Imagine the number of custom algorithms that would need to be supported. Another side-effect of having separate algorithms for each would mean additional development and deploy cycles for each new use-case. This is exactly why we started thinking of building a user scriptable decision engine - which would allow supporting new use cases with ease and flexibility.
Overall we evaluated 3 different approaches to achieve this:
1. Custom parser for our decision engine
- Considering the decision engine is expressed in an abstract language and not programming language code we can sanitize it and ensure that it’s safe even if it’s coming from an untrusted source.
- If we were to build a GUI to allow non-programmers to write the decision logic - this approach would probably be the only choice which would give us control.
- Building a grammar from scratch would mean it will have it’s own limitation with regards to flexibility. Being able to build and support things that we take for granted in a programming language like loops, iterators, data types, scope etc. would now be needed to be accounted.
- Considering this would mean an abstract language it would need learning something new from scratch.
evalis considered evil in the JS community and rightly so.
evalexecutes in the scope of your current app so using
evalfor untrusted code is a complete no-no. In our case, even when it comes to unsuspecting users it can potentially mess with the code of your main app.
- There is no way of exposing only selective scope or namespace to
eval has it’s own quirks and issues; I started exploring how we could sandbox the code that is evaluated at runtime such that we get all the flexibility of
eval but at the same time be able to protect ourselves from it’s side-effects. That’s when I stumbled upon the Node.js
vm module which is part of the standard library.
eval without it’s quirks. Internally it’s like running your code within V8 Virtual Machine contexts and at the same time allowing pre-compilation, sandboxing and contexts. Also, by default the scripts do not have access to
require and few other constructs. More importantly, you can also make available context from the global scope explicitly if needed.
- Safer than using
- Like with
- Can selectively expose aspects from the global scope.
- Checking for type safety in user input is tricky. Also, would need to accommodate for an erroneous code.
- Not suited for non-programmers or for building a GUI driven decision engine.
We still have to take a call on this - open for your thoughts? Please feel free to reach out to me or Rohith. Finally, it’s interesting to see the kind of use-cases that we are coming across while building the DCO engine. On one side we are tackling scale and on the other some ingenious scripting.