My JavaScript book is out! Don't miss the opportunity to upgrade your beginner or average dev skills.

Tuesday, July 30, 2013

dblite: sqlite3 for nodejs made easy

OK, I know, the well known sqlite3 module is cool and all the glory to it ... well, it didn't work in my case :(

The Why

node-gyp is great but it's not as portable and does not scale as I'd like to.
If you try to use sqlite3 via npm in Arch Linux ARM, as example, even if the native sqlite library is there and usable that won't work ... moreover ...
What really bothers me is that node-gyp does not update within the system as any other system package would do.
You need to rebuild, recompile, re-do everything, even if you distributed a specific linux version that trust the package manager for updates and does not want to bother users with build tasks.
This is quite common in embedded hardware and related Linux distro so I've asked myself:
why on earth I cannot simply pacman -Syu once in a while and just have automagically built for me the latest version of sqlite3, the one the whole system is using and trusting anyhow, together with any other update including the node one?

The What

The repository is here!
So here the thing: dblite is nothing more than a spawn process over sqlite-shell with piped and handled io. Anything you could write directly in sqlite3 shell will just work through this module and everything that produces a result such SELECT or PRAGMA, will be parsed only once fully flushed and asynchronously at speed-light and without blowing the memory in order to create an Array of rows where these could be either transformed into objects, or simply as Array of fields.
Here the equivalent of the first sqlite3 usage example in dblite:
// node dblite.test.js
var dblite = require('dblite');
var db = dblite(':memory:');
var start = Date.now();

db.query('CREATE TABLE lorem (info TEXT)');
db.query('BEGIN');
for (var i = 0; i < 10; i++) {
  db.query(
    'INSERT INTO lorem VALUES (?)',
    ['Ipsum ' + i]
  );
}
db.query('COMMIT');
db.query(
  'SELECT rowid, info FROM lorem',
  // retrieved as
  ['id', 'info'],
  // once retrieved
  function (rows) {
    rows.forEach(eachRow);
  }
);

function eachRow(row, i, rows) {
  console.log(row.id + ": " + row.info);
  if ((i + 1) === rows.length) {
    start = Date.now() - start;
    console.log(start);
    db.close();
  }
}
Interesting note is that in my Macbook Pro above code performs in about 4~5 milliseconds against about 15~21 milliseconds using the sqlite3 module: 3X faster!

An Intuitive API ... Like, For Real!

I'd like to do a test now: I write down some code and you think about what the code does. After that, I tell you what it does, and you'll realize it's hopefully and most likely what you thought ... deal?
db.query(
  'INSERT INTO table VALUES (?, ?)',
  [null, 'some text']
);
db.query(
  'INSERT INTO table VALUES (:id, :value)',
  {
    id: 123,
    value: "wat's up?"
  }
);
I believe you understand these are just inserts with automatically addressed and escaped values, am I correct?
Let's do something else!
db.query(
  'SELECT * FROM table WHERE id = ?',
  [123],
  function (rows) {
    console.log(rows.length);
    console.log(rows[0]);
  }
);
What do you say? A select with an id that will produce an output like this?
1 // the rows length
['123', "wat's up?"] // the row itself
OK, OK, you got that ... how about this one then?
db.query(
  'SELECT * FROM table WHERE id = ?',
  [123],
  {
    id: Number,
    text: String
  }
  function (rows) {
    console.log(rows.length);
    console.log(rows[0]);
  }
);
Would you ever bet this is the result in console?
1 // still the rows length
{id: 123, text: "wat's up?"} // the row
How about all together?
db.query(
  'SELECT * FROM table WHERE id = :id AND value = :value',
  {
    id: 123,
    value: "wat's up?"
  },
  {
    index: Number,
    value: String
  }
  function (rows) {
    console.log(rows.length);
    console.log(rows[0]);
  }
);
Yep, validation will populate the resulting row as {index: 123, value: "what's up?"} since this is how properties can be remapped in a query results: specifying object properties names adding validations to the result.
db.query(
  'INSERT INTO users VALUES (?, ?, ?)',
  [null, 'WebReflection', '1978-05-17']
);
// what can we do with that date as string?
db.query(
  'SELECT * FROM users WHERE name = ?',
  ['WebReflection'],
  {
    id: Number,
    name: String,
    bday: Date
  },
  function (rows) {
    rows[0];
    /*
    {
      id: 35,
      name: 'WebReflection',
      bday: [object Date]
    }
    */
  }
);
As summary, here is how the query method works: a SQL statement, optional fields to escape for the query, optional fields to populate results as objects instead of arrays and optional validation per each field where the default is always String.
I believe this is straight forward enough but if I am wrong please tell me your idea of intuitive API after playing a little bit with this query one, thanks :)

The Target

Raspberry Pi, Cubieboard, and other ARM based Hardware are the main tested platforms and if it goes fast there, it goes fast everywhere.
As written and tested in the main github project page, it takes 0.178 seconds for 100 inserts in a SD Card and Raspberry Pi while it takes on average 30 milliseconds to fetch 200+ rows at once and memory consumption is considered too.
I will test properly sqlite3 module performance against this one but I believe there are many cases this wrapper for a single spawn object could surprise in term of performance delegating all the horses power to the native sqlite3 shell without bindings around.

Enjoy!

Wednesday, July 24, 2013

IE8 Is More W3C Standard

I know I am probably late for this party but I am re-exploring here and there Desktop web development and developers approach to common Desktop problems.
Pointless to mention that IE8 is probably the biggest one so here I am with some good news!

W3C DOM Level 2 Implemented In IE8

You read that properly, including custom bubbling events!
This is how much this ambitious but overall slim project is about, trying to harmonize IE8 and IE8 only so that we can focus more on standard W3C DOM API without requiring any external library at all unless necessary.

Tests To The Rescue

If you don't believe it or you would like to try it in a real IE8 (no simulated) browser, here the interactive link that will work with all modern Desktop and Mobile browsers plus IE8 thanks to the mentioned library.
The test is interactive in the meaning that at some point an action, a real one from the user, is expected such a click, an input focus or an input blur so that all tests, synthetics and not, are properly tested and behaves as expected.

Readapting W3C Style

The old IE8 gotchas are still there where the most disturbing one in this case is the following:
element.addEventListener(
  'x:event',
  function problem(e) {
    element.removeEventListener(
      'x:event', problem, false
    );
  },
  false
);
Above example will fail in IE8 because of the expression bug that creates an outer scope reference to a declared function. In few words, the assigned method won't be problem but it's referenced expression.
var really = function problem() {};
// only in IE < 9
alert(typeof problem === 'function' &&
  // note these are different
  problem !== really
);
// true
Unbelievable? Well, an old gotcha already demystified where we have 3 easy solutions:
// first solution
var solved;
element.addEventListener(
  'x:event',
  solved = function solved(e) {
    element.removeEventListener(
      'x:event', solved, false
    );
  },
  false
);

// second solution
element.addEventListener(
  'x:event',
  function(e) {
    element.removeEventListener(
      'x:event', arguments.callee, false
    );
  },
  false
);
Well, yes, arguments.callee is still a thing and a very important and useful one in IE less than 9 world: go for it if you are supporting this browser!
A third solution reminded me by David is the following, based on function declaration only:
// third solution
function solved(e) {
  this.removeEventListener(
    e.type,
    solved,
    false
  );
}
element.addEventListener(
  'x:event',
  solved,
  false
);

eddy.js Is Available Too

That's correct, the eddy test page you might want to test with IE8 too should be completely green which means that IE8 could use without problems the eddy.js core library: isn't this awesome?

Thursday, July 18, 2013

eddy.js - A Bold Approach

Not the first time some project of mine ends up in JavaScript Weekly, however this time something different happened since eddy.js reached my top 5 projects list in my personal Github repo in about a week since my first tweet and I've never even blogged about that and: This. Is. Awesome!

eddy.js In A Nutshell

JavaScript is the easiest event driven programming language I know and every library out there knows this too!
jQuery, node.js, these are just the most successful JavaScript API out there and mostly based on event driven development, either with asynchronous API and libraries behind or based on DOM behavior as jQuery is for sure.
eddy.js aim is to bring the ease of an event driven application everywhere, hopefully matching everybody taste in both DOM and server side world too!

Fairy Tales ... Less!

Let's face reality: except for switching the light on or off where .switch() is most likely the most semantic and probably a better method name you could think about for such operation, we are used to think that on is a prefix for any sort of event (onload, onresize, onmouseover, onsomethinghappened) and still .on() is the method to define an event handler in most successful libraries since JavaScript time.
Moreover, switching the light on or or off is really a 1% of time edge case action in web/JS programming since the usual habit is disabled or enabled or, electricity speaking, connected.
OK, OK ... if you are programming some developer board based on relays to switch on or off things, these words could have a bigger meaning for you but seriously, in JS, and since about ever, these have been the way to understand events, and event driven behaviors!

eddy.js To The Rescue

In about 1KB minzipped, eddy.js transforms any object and only if needed into an EventTarget or, in a node.js world, EventEmitter, being absolutely as CPU and memory safe as possible, providing though enormous advantages for any kind of object!
var generic = {};
// OK, at this point I'd love to know if
// this object will ever fire a notification
// no matters what it does
// I just would like to register ... but how?
// wait a second, Hell Yeah, eddy.js!!!
generic.on('hell-yeah', notifyMe);
It's really that simple: the moment we need an event drive behavior, we are sure that any object that inherits from Object.prototype will expose those methods we all love and those will work as we expect! Isn't this much simplified than following code?
// the current node.js way
function PretendingThisIsFine() {}
PretendingThisIsFine.prototype = Object.create(
  require('events').EventEmitter.prototype,
  {
    constructor: {
      value: PretendingThisIsFine
    }
  }
);

// later on
var generic = {};
var boring = new PretendingThisIsFine();

// promoting generic now ... but wait ...
// what if it was already defined?
// this will fail like hell since all
// behaviors related to the object are lost!
generic.__proto__ = PretendingThisIsFine.prototype;
// now generic is not the object we know anymore
// this is not a solution!
So here the good news, trusting a simple 1KB library that could be served in any CDN, above scenario could simply be like this:
// the specific EventEmitter class?
// function YAGNI() {}

// later on
var generic = {};

// whenever we need it ...
generic.on(event, behavior);

Slightly Obtrusive But Totally Pragmatic !

Back in those days where libraries defined new needs for JavaScript as Prototype library and its Function#bind did, we are also in a different time where Object.prototype properties can be extended as not enumerable so that every for/in loop will be safe, how great is that?

... And What About IE8 ?

Yes, as much as I am reluctant to this topic, it's supported!
However, right now only for the JScript part, hopefully pretty soon the whole package with DOM too!
Anyway, if you are here worrying about IE8 and Object.prototype pollution and for/in problems, I've got some good news for you:
  • you are still supporting IE8, you are using obj.hasOwnProperty(key) in every loop so you are safe
  • we are in 2013, IE8 survived from 2009 until now but if today some library will start to pollute in a non enumerable way the Object.prototype that's just OK since your IE8 code should be aware of for/in problem, isn't it?
  • if you think it's too late to spend time refactoring for IE8 ... well, you nede to do that pretty soon regardless so this might be your very best occasion to adopt a simplified approach to event driven development: go for it!

Still Some IE8 DOM Gotcha

I am trying to create a way to obtain, and in way less lines of jQuery code, a unified behavior for IE < 9 so that synthetic events can propagate basically same way DOM events would do there but for any other IE8 based library, eddy.js should be just fine!

Already Compatible

If you add eddy.js after libraries that relies in a weak .on() feature detection, eddy.js will be sitting there for all other objects that did not match library criteria so that eddy.js is tested as compatible against jQuery and all other libraries.
Give it a try, please, and let me know what didn't make you happy, if anything except some old school morality about extending the Object.prototype ... I mean, there are no other ways to obtain same usefulness if not enriching the prototype root, isn't it?
Or maybe you want to go down with this chaotic pattern?

Tuesday, July 09, 2013

Some JS Descriptor Trick

I am back home for 20 minutes after 10 days of vacation in Italy and already bored so which better way than talk about some wizardish trick about JS descriptors? (yes, they were showing again the great wizard of OZ in the United Airline flight from Frankfurt ... thanks for wondering...)

The Recycling Trap

Descriptors can be recycled without problems and reused to define same things here and there. However, there is an undesired side effect about recycled descriptors: these works with constant/static values, getters, or setters, but are unable to change behavior in their lifetime.
var propertiesDescriptor = {
  shared: {
    value: Math.random()
  }
};
Above descriptor is just a basic example where Object.defineProperties({}, propertiesDescriptor) will pollute the empty object with always the same random value.
var a = Object.defineProperties({}, propertiesDescriptor),
    b = Object.defineProperties({}, propertiesDescriptor);
a.shared === b.shared; // true !

Describe A Descriptor With Descriptors

This sounds like a descriptorception and it's actually exactly that one: a property described as property descriptor, able to be different every time the descriptor is used to define properties.
Here how we could enrich the propertiesDescriptor object in order to have a runtime property too.
Object.defineProperty(
  propertiesDescriptor,
  'runtime',
  {
    enumerable: true,
    // a getter is required
    get: function () {
      // so that every time the object
      // is used as properties descriptor
      // this returned value will be used
      // as "runtime" descriptor instead
      return {
        value: Math.random()
      };
    }
  }
);
We can perform the check one mor time now against same code.
var a = Object.defineProperties({}, propertiesDescriptor),
    b = Object.defineProperties({}, propertiesDescriptor);
a.shared === b.shared; // true !
a.runtime !== b.runtime; // true again, hooray!

Non Scalar Only Values: Achievement Unlocked

This is pretty much what we have achieved with latest trick: the possibility to recycle and reuse a descriptor being sure this will hold all static/scalar/constant properties as we wanted, and also create new objects, methods, or features, each time the same descriptor is used to define one or more property.

A Basic Counter Example

Let's say we'd like to know how many times the same properties descriptor has been used during a program lifecycle, you know what I mean ? All together:
var propertiesDescriptor = {
  shared: {
    value: Math.random()
  }
};

Object.defineProperty(
  propertiesDescriptor,
  'runtime',
  {
    enumerable: true,
    get: function () {
      this.__count__++;
      return {
        value: Math.random()
      };
    }
  }
);

Object.defineProperty(
  propertiesDescriptor,
  '__count__',
  {
    writable: true,
    value: 0
  }
);

var a = Object.defineProperties({}, propertiesDescriptor),
    b = Object.defineProperties({}, propertiesDescriptor);

alert([
  a.shared,  // 0.1234
  b.shared,  // 0.1234
  a.runtime, // 0.5678
  b.runtime  // 0.8901
].join('\n'));

alert(propertiesDescriptor.__count__); // 2

Lazy Value But Not Lazy Property

The last example is about creating a descriptor with a specific property that will create a new value once the descriptor has been used with a generic object.
This time is about creating a lazy property value only when accessed through the extended object and not through the property name descriptor ... right ?
var propertiesDescriptor = {
  shared: {
    value: Math.random()
  },
  lazy: {
    configurable: true,
    get: function () {
      return Object.defineProperty(
        this,
        'lazy',
        {
          value: []
        }
      ).lazy;
    }
  }
};

// OR
Object.defineProperty(
  propertiesDescriptor,
  'lazy',
  {
    enumerable: true,
    value: {
      configurable: true,
      get: function () {
        return Object.defineProperty(
          this,
          'lazy',
          {
            value: []
          }
        ).lazy;
      }
    }
  }
);
At this point the object, let's say a generic constructor.prototype, will be described as shared accessor so that each instance, together with the prototype itself, could redefine that property only when accessed.
This is in theory better for memory usage and GC operations but hey ... I've said since the beginning these were just tricks, isn't it?
Sim Sala Bim!