Support Node.js domains

[domenic]

Some helpful comments from @mikeal on a probable strategy for supporting domains. This would help, from what I understand, in two situations:

1. Putting exceptions thrown by .done() into the current domain.
2. When an event emitter or other domain-using callback library is used inside a fulfillment or rejection handler, and throws an error, it would escape Q but wouldn't escape the domain. E.g.

return Q.resolve().then(function () {
    doSomethingAsyncWithoutPromises(function (err, whatever) {
        // Q doesn't catch this, but domains would if we wired it up right.
        callANonexistantFunction();
    });
});

[domenic]

This already works without us doing anything, I think because process.nextTick is used. @mikeal or @isaacs feel free to confirm/deny.

This test passes:

describe("node domain support", function () {
    it("should work or something", function (done) {
        var error = new Error("should be caught by the domain's handler");
        var domain = require("domain").create();

        domain.run(function () {
            Q.resolve().done(function () {
                setTimeout(function () {
                    throw error;
                }, 10);
            });
        });

        var errorTimeout = setTimeout(function () {
            done(new Error("Wasn't caught"));
        }, 100);

        domain.on("error", function (theError) {
            expect(theError).toBe(error);
            clearTimeout(errorTimeout);
            done();
        });
    });
});

[ForbesLindesay]

What about code using Q.defer()...how does that work with arbitrary resolving code.

[domenic]

@ForbesLindesay I don't quite see what you're getting at? E.g. what would be a potential test case.

Also I just realized I totally forgot bullet point 1 in my OP. It probably works though. Lemme add another test.

[ForbesLindesay]

Something like the following, where request hasn't done anything specific to be domain aware?

var error = new Error("should be caught by the domain");
var d = domain.create();

d.run(function () {
    callAsync().done();
});

var errorTimeout = setTimeout(function () {
    done(new Error("Wasn't caught"));
}, 500);

d.on("error", function (theError) {
    expect(theError).toBe(error);
    clearTimeout(errorTimeout);
    done();
});

function callAsync() {
    var def = Q.defer();
    request('https://www.google.com', function (err, res) {
        def.reject(error);
    });
    return def.promise;
}

(I haven't actually tried this)

[mikeal]

actually, after a small refactor several months ago, the calling of the request callback always happens in an event handler on the Request event emitter so it's already bound to the current domain (assuming there is one) by nature of being an event emitter. in other words it's better to think of request's callback API as an eventemitter event handler. a better illustration might be redis' callback API which works on top of a connection pool and therefor isn't automatically scoped to the current domain.

[domenic]

@mikeal is there a simple async function we can call (or setup we can create) to exhibit this behavior, without dragging in a redis library? E.g. some way I could modify the test case above to use something other than setTimeout and somehow escape the domain.

[mikeal]

you could create a little callback api around a tcp server and hit it a few times, popping the callbacks off of an array.

setTimeout and nextTick will both already get trapped.

On Nov 28, 2012, at November 28, 20129:04 PM, Domenic Denicola notifications@github.com wrote:

@mikeal is there a simple async function we can call (or setup we can create) to exhibit this behavior, without dragging in a redis library? E.g. some way I could modify the test case above to use something other than setTimeout and somehow escape the domain.

—
Reply to this email directly or view it on GitHub.

[domenic]

Would love some sample code there when you get the time; I still can't quite imagine how that works.

Since nextTick is already trapped and all Q callbacks get called in nextTick, it seems likely we're already OK. But better safe than sorry.

[isaacs]

@domenic Here's some code that will not behave properly.

// The "pool"
var EE = require('events').EventEmitter
var e = new EE
e._interval = setInterval(function() {
  e.emit('beep', Math.random())
}, 100)

// the API
function beep(cb) {
  e.once('beep', cb)
}

// the client
var domain = require('domain')
var d = domain.create()
d.on('error', function(er) {
  console.error('got an error', er)
})
d.run(function() {
  beep(function(n) {
    // we've escaped the domain!
    throw new Error('oops i accidentaly the javascript')
  })
})

[isaacs]

So, the Bad Thing only happens because the "async" call is not using any new async mechanism, but instead hopping onto the event emitted by something that was set up before entering the domain.

As a result, because callbacks are not implicitly bound (though any way to get them "into the future" IS implicitly bound, if it's created right now), the callback gets called outside the domain.

In this case, it can be solved by using an explicitly bound callback:

// The "pool"
var EE = require('events').EventEmitter
var e = new EE
e._interval = setInterval(function() {
  e.emit('beep', Math.random())
}, 100)

// the API
function beep(cb) {

  // -=-=-=--=-=-=-=-=-=-=-=-=-
  // THE FIX:
  if (process.domain)
    cb = process.domain.bind(cb)
  // -=-=-=--=-=-=-=-=-=-=-=-=-

  e.once('beep', cb)
}

// the client
var domain = require('domain')
var d = domain.create()
d.on('error', function(er) {
  console.error('got an error', er)
})
d.run(function() {
  beep(function(n) {
    // there is no escape!
    throw new Error('oops i accidentaly the javascript')
  })
})

There are two ways to look at this.

Since a lot of people use flow control libs, you can solve a lot of un-found problems by putting explicit binding in the flow control libs. (Q, async, etc.)

Otoh, that's potentially adding a cost which is unnecessary most of the time. There are not many client-pooling libs out there that re-use event emitters, and provide a callback API. It's probably easier to just patch it in the few places where it's actually broken, especially they're still broken even if the flow control libs are patched, that won't help people who aren't using Q or async.

[domenic]

@isaacs would this also be a fix?

function beep(cb) {
  e.once('beep', function () {
    var args = arguments;
    process.nextTick(function () {
      cb.apply(null, args)
    })
  })
}

If so I imagine Q is already fine as we do something similar.

In either case thanks so much for taking the time to give a nice example and talk us through the problem. I think it'll be worthwhile adding to Q (if we don't have it working already); it'd be far from the first unnecessary cost we've taken on ^_^

[isaacs]

@domenic No, it would not fix it. When you create the nextTick, you're out of the domain. nextTick will be implicitly bound to the currently active domain, but when you're calling process.nextTick, there is no currently active domain, because you're in the context of the e object, which does not have a domain.

[ForbesLindesay]

So the test case (which we suspect fails) could look like:

var error = new Error("should be caught by the domain");
var d = domain.create();

var e = new require('events').EventEmitter

d.run(function () {
    callAsync().done();
});
e.emit('beep');

var errorTimeout = setTimeout(function () {
    done(new Error("Wasn't caught"));
}, 500);

d.on("error", function (theError) {
    expect(theError).toBe(error);
    clearTimeout(errorTimeout);
    done();
});

function callAsync() {
    var def = Q.defer();
    e.once('beep', function () {
        def.reject(error);
    });
    return def.promise;
}

P.S. I hadn't been referring to @mikeal's request library, but rather a generic, badly behaved request method that retrieved something from a web-server.