Monday, May 29, 2017

File deduplication written in bash

Once there was this guy asking me would I be able to write a file deduplication script in shell.

It is not very hard and it is a curious problem, so I am publishing my code here:

#!/bin/bash
[ ! -d $1 ] && echo "$1 is not a directory! exit" &&  exit 1
cd $1
oldsize="yyyyy";oldname="xxxxx"
find . -type f -ls | awk '{ print $7":"$11 }' | sort -k 1,1 -n -r | while read line; do
  size=${line%%:*}
  name=${line##:*}
  if [ "$oldsize" == "$size" -a -f "$name" -a -f "$oldname" ] && diff -s "$oldname" "$name"; then
      rm -f "$name"
      ln "$oldname" "$name"
      continue
  fi
  oldsize="$size"
  oldname="$name"
done
I am wondering, would it be possible to be made even simpler...

Monday, April 3, 2017

UNIFICli - a CLI tool to manage Ubiquiti's Unifi Controller

As mentioned earlier, I made a nodejs library interface to the Ubiquiti Unifi Controller's REST API which is available here - https://github.com/delian/node-unifiapi

Now I am introducing a small, demo, CLI interface, which uses that same library to remotely connect and configure Ubiquiti Unifi Controller (or Ubiquiti UC-CK Cloud Key).

This software is available on GitHub here - https://github.com/delian/unificli and its main goal for me is to be able to test the node-unifiapi library. The calls and parameters are almost 1:1 with the library and this small code provides great example how such tools could be built.

This tool is able to connect to a controller either via direct HTTPS connection or via WebRTC trough Ubiquiti's Unifi Clould network (they name it SDN). And also you have a command you could use to connect to wireless access point via SSH over WebRTC.

The tool is not completed, neither have any goal. Feel free to fix bugs, extend it with features or provide suggestions. Any help with the development will be appreciated.

Commands can be executed via the cli too:
npm start connectSSH 00:01:02:03:04:05 -l unifikeylocation

Tuesday, March 14, 2017

node-unifiapi - NodeJS API to access Ubiquiti Unifi Controller API

I have completed the initial rewrite of the PHP based UniFi-API-Browser to JavaScript for NodeJS.
The module and some auto generated documentation with some basic examples is available here:

The supported features includes all of the UniFi-API-Browser calls via the direct REST http calls or via WebRTC (using Ubiquiti Unifi Cloud) as well as SSH access to Unifi Access Points via WebRTC.

Tuesday, January 17, 2017

Private Properties with JavaScript (prototypes)

Some are saying that JavaScript does not have private properties with prototypes and it is not like TypeScript or Python.

However, that is not fully correct, especially with this comparison.

Python doesn't have anything private. All the scopes are globally addressable and all the variables, properties and methods are accessible globally (as long as you know the scope path, which could be easily traced from within the software itself).

TypeScript is just a pre-processor. It will do extra semantic checks during compilation, but no enforcement of anything on runtime. 

However, it is generally untrue that you have no private properties available in JavaScript.

You do have scope inheritance and you don't have global access to any scope from outside. Therefore you can make private properties easily.

For example, if you have:

function F1() {
  var XXX = 0;
  return function () {
     return XXX;
  }
}

F1()() will respond with the value of XXX but there is no way to access XXX from outside.

So what about prototyping?
You can do this there too.

function F1() {
  var XXX = 'yyy'; // Private property
  function F1() {
     // You constructor is here
     this.YYY = 'xxx'; // public property
  }
  F1.prototype.method1 = function() {
    return XXX + this.YYY;
  }
  return new F1();
}

Then if you do:

x = F1();
or
x = new F1();  // Both works the same way
and check:
console.log(x.YYY); // output 'xxx'
console.log(x.XXX); // output Error
console.log(x.method1()); // output 'yyyxxx'

So everything works! You have prototyping with private and public properties.
The same way you can do private and public methods.

The only thing that will not work with this method is instanceof
x instanceof F1 will respond with false, because x is not an instance of the upper F1, but the inner F1

The current ES6 standard have classes, but they are essentially a wrapper on top of prototype. So this technique still apply. Additionally with Object.defineProperty you could apply extra protection on top of a property.

There is another approach to the problem. Without prototypes.

function F1() {
  if (!this instanceof F1) return; // Protect against global scope polution
  var XXX = 'yyy'; // Private property
  this.YYY = 'xxx'; // Public property
  function PrivateMethod() {
    return XXX + this.YYY;
  }
  this.method1 = function() {
    return PrivateMethod();
  }
}

With the shown above approach you have both private and public methods and properties, with workable instanceof

Monday, January 2, 2017

Python UniFI API for management of Ubiquity UniFI infrastructure

I am porting to Python the UniFi Browser API (PHP) which could be used to manage the UniFI AP infrastructure from external application to the UniCK Controller.

The port is completed, however, the work is ongoing, the API is updated (adding new calls and support for HotSpot 2.0) and changes may happen.

Documentation will come a bit later, but I guess the code is self explanatory.

For those who are interested, you could find it here:

Sunday, October 4, 2015

Passport-http Digest Authentication and Express 4+ bug fix

Passport is quite popular environment for implementing authentication under Node.JS and Express Framework.

Passport-http is a plug-in that provides Digest Authentication interfaces and is very popular for Passport.

However, since Express 4, the default approach to the Express routes is them to be relative.

Example - 
Express <=3 default application assumed that every file who extend Express with routes should specify the full path to the route in a similar to this approach:

app.js:
app.get('{fullpath}',function)...

Or:
require('routes/users')(app)

Where users.js do as well:

app.get('{fullpath',function)...

The default app of Express 4+ uses relative routes, which is quite better as it allows full isolation between the modules of the application.

Example:

app.js:
app.use('/rest',require('routes/rest'));

Where rest.js has:

router = require('express').Router();
router.get('/data', function)... // where data is relative url and the actual will be /rest/data

This simplifies the readability. Also it allows you to separate the authentication. You could have different authentication approach or configuration to each different module. And that works well with Passport.

For example:

app.js:
app.use(passport.initialize());
app.use(passport.session());

rest.js:
var DigestStrategy = require('passport-http').DigestStrategy;
... here there should be a code for authentication function using Digest ...

and then:
router.get('/data',authentication,function) ....

This simplifies, makes it more readable and isolates very much the code necessary for authentication.

Personally, I write my own authentication functions in a separate module, then I include them in the express route module where I want to use them and it became even more simpler:

rest.js:
var auth = require('../libs/auth.js');
Router.get('/data', auth('admins'), function) ...

I even could apply different permissions, roles like - if you have pre authenticated session, then the interface will not ask you for authentication (saved one RTT) but if you don't it will ask you for digest authentication. Quite simple and quite readable.

However, all this does not work with Passport-http, because of a very small bug within.

The bug:

For security reasons, passport-http module verifies that the authentication URI from the customer request is the same as the URL requested authentication. However, the authentication URI (creds.uri) is always full path, but it is compared to req.url which is always relative path. The comparision has to be between creds.uri and req.baseUrl+req.url.

And this is my fix proposed to the author of passport-http, which I hope will be merged with the code.

Friday, September 25, 2015

EmbeddedJS - async reimplementation

I like using Embedded JS together with RequireJS in very small projects. It is very small, lighting fast and extremely simple, powerful and extensive.

However, there are hundreds of implementations, most of them out of date, and particularly the implementation I like is not supported anymore. But that I mean not only that the author is not supporting it, but also it works unpredictably in some browsers because it rely on Sync XMLHttpRequest which is not allowed in the main thread anymore.

So I decided to rewrite that EJS implementation myself, in a way I could use it in async mode.

So allow me to introduce the new Async Embedded JS implementation, which is here at https://github.com/delian/embeddedjs

It supports Node, AMD (requirejs) and globals (as the original) and detect them automatically.
A little documentation can be found here https://github.com/delian/embeddedjs/blob/master/README.md

The new code is written in ES5 and uses the new Function method from ES6. That makes it working only in modern browsers (IE10+). But it makes it really really fast. By current estimates about twice faster than the original.

It is still work in progress (no avoidance of cached URLs for example), but works perfectly fine for me.

If you use it and hit a bug, please report it at the Issues page at Github