FILTER.js

A pure JavaScript Library for Image/Video Processing, Filtering and Computer Vision

This is a library for processing images/video in pure JavaScript using HTML5 features like Canvas, WebWorkers, WebGL and SVG (in progress) or analogs in Node.js.

version 0.9.6

• filter.core.js / filter.core.min.js
• filter.io.js / filter.io.min.js
• filter.codecs.js / filter.codecs.min.js
• filter.filters.js / filter.filters.min.js
• filter.plugins.js / filter.plugins.min.js
• filter.bundle.js, filter+io+codecs+filters+plugins+dependencies

see also:

• Contemplate a light-weight template engine for Node/XPCOM/JS, PHP, Python, ActionScript
• HtmlWidget html widgets used as (template) plugins and/or standalone for PHP, Node/XPCOM/JS, Python (can be used as plugins for Contemplate engine as well)
• Tao A simple, tiny, isomorphic, precise and fast template engine for handling both string and live dom based templates
• ModelView a light-weight and flexible MVVM framework for JavaScript/HTML5
• ModelView MVC jQueryUI Widgets plug-n-play, state-full, full-MVC widgets for jQueryUI using modelview.js (e.g calendars, datepickers, colorpickers, tables/grids, etc..) (in progress)
• Dromeo a flexible, agnostic router for Node/XPCOM/JS, PHP, Python, ActionScript
• PublishSubscribe a simple and flexible publish-subscribe pattern implementation for Node/XPCOM/JS, PHP, Python, ActionScript
• Regex Analyzer/Composer Regular Expression Analyzer and Composer for Node/XPCOM/JS, PHP, Python, ActionScript
• Xpresion a simple and flexible eXpression parser engine (with custom functions and variables support) for PHP, Python, Node/XPCOM/JS, ActionScript
• GrammarTemplate versatile and intuitive grammar-based templating for PHP, Python, Node/XPCOM/JS, ActionScript
• Dialect a simple cross-platform SQL construction for PHP, Python, Node/XPCOM/JS
• Abacus a fast combinatorics and computation library for Node/XPCOM/JS, PHP, Python, ActionScript
• Asynchronous a simple manager for async, linearised, parallelised, interleaved and sequential tasks for JavaScript
• RT client-side real-time communication for Node/XPCOM/JS with support for Poll/BOSH/WebSockets

Contents

• Live Examples
• Browser Support
• Credits
• References
• Features
• API Reference
• ChangeLog
• Todo

Live Examples

• Image Processing with Filter.js
• Video Processing with Filter.js
• Sound Visualization with Filter.js (Trioptic)
• Filter.js with Three.js
• Filter.js in Node.js

Browser Support

Credits

Some filters code has been adapted from open source libraries (mostly c, java and flash, plus a couple from javascript libraries), see the comments in the code for details.

• Image Processing Library in Java
• AS3 Image Processing Library
• AS3 ColorMatrix by @GSkinner
• Simplex Noise and Perlin Noise by Stefan Gustavson
• glfx.js
• JViolaJones, HAAR.js
• OpenCV, HAAR cascades
• zlib (asm.js emscripten version)
• ffmpeg (asm.js emscripten version)

Some image processing/computer vision theory, basics and tutorials (see references):

• light, color, perception and color space theory
• the influence of history & culture on visual perception
• a beginners guide to bitmaps by Paul Burke
• OpenCV, open source computer vision
• General-purpose GPU Scientific Computing (..moving towards)

Features

The library dependencies are:

• Classy.js micro Object-Oriented framework.
• Asynchronous simple manager for async/parallel tasks.

The framework defines an Image Proxy class, which represents an Image, a number of utilities like Color Class, Image Loader classes, Image Codecs, and 17 generic Filter types (some having glsl/svg analogs) plus various Plugins and Extra filters (with support for parallel processing transparently both for browser and nodejs)

0. AbstractFilter
1. ColorTableFilter
2. ColorMatrixFilter (analogous to the ActionScript filter)
3. ColorMapFilter
4. AffineMatrixFilter
5. GeometricMapFilter
6. DisplacementMapFilter (analogous to ActionScript filter)
7. ConvolutionMatrixFilter (analogous to the ActionScript filter)
8. MorphologicalFilter
9. StatisticalFilter (previously called NonLinearFilter)
10. BlendFilter
11. CompositeFilter (an abstraction of a container stack for multiple filters)
12. AlgebraicFilter (an abstraction of algebraic combination of images or other filter outputs into an output image, to be added)
13. InlineFilter (create inline filters dynamicaly at run-time using your custom functions)
14. DimensionFilter
15. GLSLFilter glsl-based (webgl/node-gl) analogs of at least some of the generic filters (in progress, possibly in next update)
16. SVGFilter svg-based filters (todo)
17. Plugins (a number of plugin filters which cover a wide(r) range of functionality and use cases)

Each of the generic filters is prototype but it also includes a number of implementation filters like grayscale , colorize , threshold , gaussBlur , laplace , emboss , gamma, twirl and so on.. (depending on type of filter)

Parallel Processing Support (browser and node) (support parallel procesing/filtering with filter workers in an intuitive and transparent way, see examples)

GPU Processing Support (browser and node, in progress) (support GPU-based parallel procesing/filtering with glsl filters in an intuitive and transparent way)

Image Blending Modes (analogous to PhotoShop blend modes)

The filters, and the way they operate, naturaly represent a system of interconnected nodes which process and interchange (image) data (not necesarily synchronously), a.k.a a signal processing graph system. The result is a streamlined flow for image processing and computer vision in JavaScript.

TIP: You can create your custom build of the library with the filters/plugins you choose.
Each filter and plugin is independent and can be used in a mix-n-match manner, as long as the core classes are always included.
Change the dependencies file(s) to include your own selection of filters and plugins for your custom build

Todo

• add GLSL (webgl/node-gl) support for various generic Filters (in progress, possibly in next update)
• add some needed signal processing graph node filters (eg algebraic, switch, delay etc..) (in progress)
• add active-shape geometric filters, color/histogram-detector filters, .. (todo)
• add 2d-fft routines, frequency-domain filtering (todo)
• add SVG, CSS Filters interface support for some Filters (todo)
• add machine learning (image) segmentation/clustering algorithms (e.g kmeans, kmedoids, connected components, deterministic annealing, svd, jade, ..) [DONE partially]
• implement some numeric routines (e.g blas, filter routines) using faster asm.js (browser & nodejs) and/or simd.js [DONE partially]
• make convolutions/statistics faster [DONE partially]
• add full support for Node.js [DONE]
• add (generic/native) codec support for image formats, e.g .TGA, .HDR/.RGBE, .GIF, .BMP, .PNG, .JPG/.JPEG etc.. [DONE]
• add support for Parallel Processing using Web Workers and/or Asynchronous Processing [DONE]
• use fixed-point arithmetic, micro-optimizations where possible [DONE]
• add caching of filter parameters where applicable [DONE]
• increase performance for Opera, IE [DONE partially]