Building a news aggregator module for a trading platform -Introduction

I am working on a project where I going to build a news aggregator which takes certain keywords and searches a predefined set of websites, company statements etc for stories or blog posts that have that keyword in it. The keywords are mostly a combination of company names or stock ticker codes and mood indicators, people’s names, hedge fund manager comments etc.

The key differentiator of this project than other news aggregators (of which there are hundreds) is that this allows a configurable set of relationships and correlations to be defined to represent the trader’s world view. The news then streams in and is matched against these relationships to enable a challenge or proof in real time of that view.

The ultimate purpose of this tool is to guide traders in placing trades and making investments by using this up to date and very focused picture of global events.

I thought this would make a great series of blog posts to discuss the architecture, code patterns and methodology used in building this very cool tool. Also when the tool goes to general market, this will be a log of the quality that has gone in to the tool to raise confidence in it as a day to day tool in the traders armoury.

Being a Microsoft guy I will of course use C# to build the application. I’ll use WPF for the GUI and try to build the application using the best practices that I know.

The projects I work on generally have the following life cycle:

• A requirement or need is identified by one of my clients
• Discussion takes place to identify what the need really is, and how this can be best serviced by a technological solution if appropriate
• A proof of concept is built which uses best practices as though it was production code.
• This POC is then used to build out the application in an iterative way
• Each iteration generally lasts from 2 – 4 weeks and has the following parts:
  • Discussion with the client determines the priority of each of the functional items
  • The highest priority items are scoped and planned into the next iteration
  • User Acceptance Tests are written by the Client or Tester if we have one
  • Developers use TDD (tests first) to develop the code
  • Clients are involved with team meetings answering questions as they arise
    (This reduces developer decisions on business items and removes gold plating)
  • Developers show case the work they have done to client
  • Feedback then gets either built in, or scheduled for re-prioritization next iteration
  • UAT  happens
  • Application changes are rolled to production for the client to start using it

They also have the following key elements:

• All team members having access to ask questions to the client and vice versa
• Developers all use the same ubiquitous language as the client
• A rich domain model built on top of the common language
• Every team member sharing responsibility of make working software on time
• Commitment of everyone to deliver working software that is fit for purpose
• Dedication to best practices including TDD
• Sustainable development
• Clear communication of what is being delivered and when
• Trust built by the development team from the client who can clearly see the above steps taking place

This creates an exciting enthused environment for achieving success. This is the environment we will be building our news aggregator. Next post will cover the general code design.