My experience


Self-EmployedApril 2019 - Present

Currently working on my own projects and also a proposal for the RSSB Data Sandbox+ competition - this project will investigate using machine learning to identify and categorize delays within the real-time Network Rail data feeds. Also, on a part time basis, finishing up designing and building a Hadoop / DVC / MLFlow based machine learning framework for UEA.

University of East AngliaMarch 2018 - April 2019

The senior researcher on a RSSB sponsored / UEA led feasibility study investigating how machine learning (neural networks and ensembles), expert systems, and genetic algorithms could be used to improve the management of train delays. Responsibilities have included:

* Design / development of
- the AIEDSS system architecture
- the framework for running machine learning experiments
- a ETL pipeline for processing Network Rail datasets
- a Apache Hive / Drill / Tez / HDFS Hadoop cluster
- a Neo4j database that stores delay propagation graphs
- a novel data visualisation tool for analysing the delay graphs

* Research into models and ensembles for best predicting train running times

* TA for the AI class, assisting students with development of a chatbot using NLP / expert systems

* Writing documentation for the project (technical reports, drafts of the research papers)

Independent ContractorOctober 2017 - March 2018

Projects worked on:
1) extending the results of my dissertation with an eye towards publication
2) a system for automatically categorizing bookmarks / web pages through clustering and text analysis
3) a system for analyzing novels in order to gauge their literary 'quality', if only to provide editors some initial guidance on what to read first in the slush pile - or provide insights to authors on how they could improve their writing

University of East AngliaOctober 2016 - September 2017

Recent graduate (MSc with Distinction) - my dissertation (winner of Comp Sci department's "Best dissertation" prize) was on developing machine learning ensembles to improve the detection of fake reviews, or spam reviews, on sites such as Amazon and TripAdvisor. Cognitive linguistics and natural language processing (NLP) are major aspects.

Projects have included research into using machine learning to evaluate the credibility of websites and the information provided as well as a information visualization system for analyzing the network graph of reviews, reviewers, and products for sites like Amazon.

Was a Teaching Assistant for several classes while working on extending my dissertation with an eye towards publication.

Proxama LLCMarch 2014 - October 2015

Senior Engineer in the Research and Development department of Proxama. Various projects worked on include:

* research into iBeacons / Eddystone and development of mobile (Android and iOS) apps to test performance, interoperability, etc of phones and beacons, including an investigation into Android Lollipop's new features. This effort also involves development of a Python based library for creating Excel based graphs and charts from the raw data.

* development of BLED112 firmware for secure versions of iBeacons and Eddystone beacons

* research into data science topics, data visualization, machine learning with an eye towards a recommendation system to process the data coming into and gathered by Proxama as part of their proximity marketing platform for more effective targeting of voucher and coupon delivery

* development of a Python library / Django app for integrating with Proxama's mobile phone apps and web based loyalty card platform

* research and development of Proxama's tokenized payments platform (Django based) for making EMV compatible payments with a mobile phone w/o needing a secure element like a SIM card

* research and development of the J2EE and Django based version of the payment platform compatible with the new VISA and Mastercard standards for HCE payment systems (was Team Lead for the Web Services functionality in the system as well as the Certificate Authority subsystem)

Microcinema InternationalJanuary 2002 - January 2014

Lead developer / system architect (part-time) in charge of developing, maintaining Microcinema's websites and system administration as well as advising the CEO on IT matters.

One major project involved overhauling the website to use the Django framework and to support Video on Demand and a new shopping cart with

Other projects included developing a system to manage royalty payments, overhauling the database schema, integration with NetSuite, Paypal and other e-commerce systems, developing a tool for non-technical users to process DVD metadata using XLS files and transferring the server across different hosting companies.

LinescapeMay 2008 - April 2013

Founding engineer / system architect in charge of system design, development, QA and management of part time contractors for Tarisoga and its public website, Duties also included advising the directors on technical matters.
Notable projects included:

* a graph database in Neo4j for holding the vessel schedules

* a Web Service for customers to access the database

* customized search functions for customers implemented using AWS's RDS

* an ETL pipeline using web scraping to get schedules from over 300 carriers that processes HTML, XLS, PDF, XML, CSV, and EDI based data.

Independent ContractorJanuary 2002 - January 2008

--- Ventures, Etc ( ---

Lead designer of the BEA Weblogic J2EE back end web application for online racetrack gambling that accepts betting data and processes it before forwarding it to other systems

Replaced the previous designed for C++ code / application that was jammed into Weblogic with a properly designed J2EE architecture -- Was rehired to port, redesign, and enhance the back end when Xpressbet moved to using JBoss

Installed and administered the Weblogic servers; duties included packaging and deployment of the application as well as development

Contractor for these websites assisting in all aspects of development:
--- & ---
--- & ---
--- ---

Cohera (PeopleSoft)December 1999 - April 2002

Cohera was a startup founded by Michael Stonebraker that was designing the first federated database -

I was a senior developer in the QA department responsible for designing and developing automated test systems for junior employees to use in their duties

* Designed / developed a scriptable Java based multithreaded automated test harness (based on JUnit) for Cohera’s web scraping software that used multiple web servers and DBMSes

* Designed and developed a Java based automated test harness for testing the various components of Cohera’s J2EE n-tier catalog management and content integration system

* Developed a library of SilkTest and SQA Robot test routines for the Weblogic based Catalog Management System to alleviate the QA department’s workload of creating test scripts

* Maintained and extended the existing Java based multithreaded automated test system that tested the Cohera Hub, a front end for integrating disparate DBMSs into a large distributed database system
Developed Perl scripts for easily installing and updating standardized test databases

* Led integration of Cohera's UNIX based QA system into Peoplesoft's Windows development environment after Peoplesoft bought Cohera in 2002.

SAICMarch 1994 - December 1999

SAIC is a contracting company for the US defense industry, so I had a TS security clearance.

* Received several out-of-cycle raises, bonuses, and a promotion for outstanding performance and productivity. In one performance review, my supervisor stated "I consider Doug one of the top two software developers in my group and one of the top five in the Operation."

* Technical lead for the Demand Assigned Multiple Access satellite communications C++ system which allowed users to dynamically create and modify satellite communication links between network terminals.

* Technical lead for the Interim Tactical Orderwire System, a multithreaded client/server C++ text and voice orderwire system used in remote satellite terminals for communications management purposes. Some notable achievements on this project included the following

- Saved a contract that was at risk of being rejected by working closely with a frustrated customer

- Developed an automated testing system for exercising the GUI, network communications layer, and the customized memory management subsystem

- Designed and implemented a reliable multicast satellite communications protocol as TCP/IP was not usable along with a protocol for properly replicating the server’s database at the client sites

* Software architect for an multithreaded SAIC war dialer that has advanced features such as scanning using multiple modems, a remote control capability, and an automated “intelligent” break-in capability

* Participated in several projects that assessed the security of commercial customers' computer systems; wrote automated scripts for those purposes and acquired a working knowledge of firewalls, voice mail hacking, Internet and website security, and various security packages

* Participated in the design and development of a Nexpert Object-based expert system for use in a device that is capable of exercising intelligent control of jamming resources to minimize ‘communications fratricide’

Pragmatics, Inc.June 1992 - March 1994

System architect and lead developer for one component / box of two satellite networking projects for the US Department of Defense

* Technical lead in the design, development, and testing for the Object Oriented Store and Forward Message Processing System (SFP). The SFP is a hub and router for messages from other components of the Secure Survivable Communications Network (SSCN), a distributed satellite communication network

* Designed a networking algorithm to prevent duplicates messages and to prevent messages from flooding the subnetworks connected to the SFP

* Rewrote low level hardware interrupt routines of the SFP’s OS for greater efficiency

* Designed and wrote a C++ library for controlling satellite modems and an OO database for other team members working on a specialized demand assigned bandwidth communications network

* Developed an OO database as well as the database schema for DABS, a specialized demand assigned bandwidth packet communications network that allows users to dynamically configure connections

* Generated system and unit test plans, software design documents, functional requirements, and interface specifications for these projects based on the DoD 2167A standard

* Responsible for the white box testing of the GUI and business logic modules of DABS as well as the unit and integration testing of the SFP

My stack

XSLT, WebLogic, Web Scraping, Unix Shell Script, UNIX, UI Design, System administration, Software Architecture, Software Architect, Quality Assurance, Python, PHP, OpenVPN, OpenSSL, OOP, Nginx, NetBeans, Neo4J, Natural Language Processing (NLP), Machine Learning, LAMP, JUnit, JBoss, Java EE, iOS, iBeacon, GraphQL, Graph Database, Git, Exim, Dovecot, Django, DevOps, Debian, Database Design, Data Science, CSS, AWS, Ant, Android development environment