Central Stock Price Information

A proposal for Season of KDE 2010 by Brian Cappello
Mentors: Cristian Onet and Alvaro Soliverez

Use Cases

A plasmoid wants to display some stock quotes stored in a watchlist.
Santosh Kumar wants to provide notification support for changes in stock prices.
An investor wants to chart an investment's yearly performance starting from their first purchase thirty-five years ago.
An investor wants to do some fundamental research on a stock they're considering trading.
A day trader wants to chart today's 5-minute price action since the open.
A swing trader wants to chart two months worth of hourly data.
Another swing trader wants to chart 200 bars of daily data.
A student of the markets wants to "replay" the crash of '87 starting from 3 months prior and advancing one day at a time.
A mechanical systems trader writes a scripted trading algorithm, and they want to back-test its performance on various sets of historical data.

[Jump to Top]

Tentative Schedule: WIP/subject to change

May W4:

Continue research into DBus, asynchronous transfers, doctools, unit tests and cmake.
Get some DBus code working!

June W1:

Attempt to "finalize" class/object separation and interaction planning.
Preliminary DBus symbol management. Add/remove only, no watchlists yet.
Implement Yahoo historical data interpreter and runtime-only storage. (Concentrate on daily data. W/M/IntraDayHistory later.)

June W2:

Implement Yahoo intraday quotes support and runtime merging/storage into most-recent-index of daily historical data.
Get on-demand DBus intraday support working.
Figure out how best to store data to HDD. Possibly as flat-file CSVs and/or KConfig?
Get SymbolManager storage-to/from-disk working.

June W3:

Get intraday quotes monitoring working. Broadcast notification of updates to clients.
Implement IntraDay quotes test cases.
Get preliminary on-demand DBus CompanyInfo/Stats working (assume client knows what keys are available).

June W4:

CompanyInfo Test cases.
Begin to get HDD saving/loading historical data working (maybe merging/updating).

July W1:

Finish up historical data HDD stuffs, including daily, weekly, monthly, and merging/updating.

July W2:

Begin work on DBus interface for historical data. No IntraDayHistory support yet.

July W3:

Finish DBus interface for historical data, including time conversion and adjusted prices.
Finish/polish Previous-Close convenience function.

July W4:

Implement Historical Data and Previous-Close test cases.
Historical Data performance profiling and optimization.

August W1:

Add DBus WatchList support.
Watchlist test cases.
Polish CompanyInfo/Stats DBus interface, add support for querying, key-name lookup, etc.

August W2:

Saving/loading of CompanyInfo/Stats to disk.
Improve/update CompanyInfo test cases.

August W3:

Polish documentation, test cases, and further profiling.

August W4:

Buffer

Time Permitting:

Expand CompanyInfo by parsing data from Yahoo's Key Statistics page.
Improved intraday real-time support by parsing data from finance.yahoo.com (or other source?)
User-controlled CSV export.
Add/finish support for IntraDayHistory.
IntraDayHistory test cases.

[Jump to Top]

Planned Components

Always-On/On-Demand Backend: A member of Alkimia

The backend handles all quote retrieval/storage, and provides a public interface for client applications to query over DBus for whatever information they need. The backend also serves as a centralized place for consolidating various stock-price-manipulation maths common to financial applications' needs.

A "Master" Backend UI: Some form/combination of plasmoid/application/kpart/kcm...

Cumulative functionality should support something along the lines of:

Managing which symbols and watchlists the backend permanently stores data for.
Configuring the backend's default historical and intra-day data sources. (And maybe resource/performance settings.)

YahooCSV: Download quotes from Yahoo! (Default for Intra-Day & Historical)

Directory: Import/monitor quotes from a directory containing CSV files. (For users with paid-data services.)

Time Permitting: YahooWEB: Parse real-time data from the finance.yahoo.com website. (Intra-Day only)

Exporting user specified data/information/statistics to CSV.

[Jump to Top]

Planned Backend Functionality

Internals

Maintain/store a master list of all watchlists.
Maintain/store a master list of all symbols to be watched. Filters out watchlist-duplicates and includes symbols which aren't a member of any watchlist.
Maintain a local/offline store of downloaded data:

Company info and statistics. (Used for offline mode/throughout the day. Company stats change frequently enough and are reasonably cheap (fast) to download so we update this once daily, if possible.)
Accumulate sampled Intra-Day data as Minutely data.
Store and keep up-to-date "primary" data: Minutely, Daily, Weekly, and Monthly Time Periods

[Jump to Top]

Public DBus Interfaces

Management of stored symbols and watchlists

These are primary functions.

Symbol Management

Add/Remove Symbols

Inputs:

Symbol
Action: Add or Remove

Output:

Bool Success

Modify a watchlist's symbols

Inputs:

Symbol
Watchlist
Action: Add or Remove

Output:

Bool Success

Watchlist Management

Get Watchlists

Inputs:

Output:

List of all watchlist names

Get Watchlist Symbols

Inputs:

Watchlist name

Output:

List of the watchlist's symbols

Add/Remove Watchlists

Inputs:

Watchlist
Action: Add or Remove

Output:

Bool Success

Lower Priority: Merge Watchlists

Inputs:

First Watchlist
Second Watchlist
Action: Merge or New

Merge: First into second, delete first. [Mirroring 'mv' command.]

New: Create a new watchlist, preserve originals.

If Reqd: New Watchlist's Name

Output:

Bool Success

[Jump to Top]

Intra-Day/Most-Recent-Close Quote

This is a convenience function.

Inputs:

Symbol
Optional: Prices or Percents

Prices: the real prices (Default)

Percents: Percent changes from previous close

Output:

struct: Contains a KDateTime, the Open, High, Low, Close, Volume, and, if applicable, Open Interest (eg for futures). Optional: and Beta.

Previous Day's Close Quote

This is a convenience function.

Inputs:

Symbol
Optional: Prices or Percents

Prices: the real prices (Default)

Percents: Percent changes from previous close (eg two closes ago)

Output:

struct: Contains a KDateTime, the Open, High, Low, Close, Volume, and, if applicable, Open Interest (eg for futures). Optional: and Beta.

Company Information, Key Statistics and Fundamentals

This is a primary function.

Inputs:

Symbol
Parameters (keys)

Initially most (all applicable) parameters easily downloadable from Yahoo will be supported. Time permitting, more information sources can be added (such as parsing parameter values from Yahoo's Key Statistics webpage).

Time Permitting: There should probably be a way to query at runtime what keys are available, as well as key-lookup by human-readable names.

Output: Must be able to accommodate both strings and numbers.

A container object holding the requested parameters' key-value pairs.

[Jump to Top]

Historical Quotes

This is a primary function.

Inputs:

Symbol
Amount/duration of historical data:

Optional: Specify an end date. (Defaults to the most recent close.)

Specify a duration (relative to the end date) by providing one of the following:

A start date.
An elapsed quantity and unit of time, independent of the (optionally) requested time period.
Or directly as a number of bars.

Optional: Time Period: Specify by providing:

Optional: A major time unit. (Daily (Default), Weekly, or Monthly)

Optional: A multiplier. Default = 1

Optional: Adjusted or Absolute Prices

Adjusted: (Default) Prices adjusted for splits and dividends.

Absolute: Raw price data. (Provided by default if the requested symbol has no splits or dividends.)

Optional: A maximum lookback. Default = 0

The number of "hidden" bars (older than the start date) to return. Used by technical analysis indicators so their results will be complete for the requested length of "shown" price data.

Output:

A container object holding the input parameters used to produce the data, as well as the data itself (containing the same information as Intra-Day structs). Need to look into what would be the most convenient storage method for MVC-based clients such as KDChart as used by KMyMoney, but that's still flexible enough to be non-MVC-friendly.

[Jump to Top]

Future Backend Possibilities

Time Permitting / Low Priority:

Support for calculating relative price/performance between multiple data sets.
Support for calculating "custom Market Indexes" from watchlists.
Support for storing/browsing/querying sectors and industries, as well as tracking their performance using (1) and (2).
Improved support for options, futures, and FOREX.
[Maybe] Technical Analysis Indicator Support: Essentially this would be providing a Qt-like API wrapped around TA-Lib's Abstract Interface. Interface documentation here. If this were to be done, it would be as an optional plugin, detected at runtime.
[Depends on TA-Lib] Manual calculation of Beta with controllable inputs for the time period and the "volatility base market." This ability is probably most applicable for large private investors, portfolio analysts, and institutions/banks, and is particularly relevant for those invested in international/emerging markets who do not want to base their volatility calculations off of American markets, as Yahoo's provided Beta is assumed to do.

[Jump to Top]