User:Bpeller: Difference between revisions

    From KDE TechBase
     
    (10 intermediate revisions by the same user not shown)
    Line 2: Line 2:
    ''A proposal for Season of KDE 2010 by Brian Cappello''<br>
    ''A proposal for Season of KDE 2010 by Brian Cappello''<br>
    ''Mentors: Cristian Onet and Alvaro Soliverez
    ''Mentors: Cristian Onet and Alvaro Soliverez
    == Use Cases ==
    == Use Cases ==
    :*A plasmoid wants to display some stock quotes stored in a watchlist.
    :*A plasmoid wants to display some stock quotes stored in a watchlist.
    Line 16: Line 17:




    == Components ==
    == Tentative Schedule: WIP/subject to change ==
    === May W4: ===
    :*Continue research into DBus, asynchronous transfers, doctools, unit tests and cmake.
    :*Get some DBus code working!
    === June W1: ===
    :*Attempt to "finalize" class/object separation and interaction planning.
    :*Preliminary DBus symbol management. Add/remove only, no watchlists yet.
    :*Implement Yahoo historical data interpreter and runtime-only storage. (Concentrate on daily data. W/M/IntraDayHistory later.)
    === June W2: ===
    :*Implement Yahoo intraday quotes support and runtime merging/storage into most-recent-index of daily historical data.
    :*Get on-demand DBus intraday support working.
    :*Figure out how best to store data to HDD. Possibly as flat-file CSVs and/or KConfig?
    :*Get SymbolManager storage-to/from-disk working.
    === June W3: ===
    :*Get intraday quotes monitoring working. Broadcast notification of updates to clients.
    :*Implement IntraDay quotes test cases.
    :*Get preliminary on-demand DBus CompanyInfo/Stats working (assume client knows what keys are available).
    === June W4: ===
    :*CompanyInfo Test cases.
    :*Begin to get HDD saving/loading historical data working (maybe merging/updating).
    === July W1: ===
    :*Finish up historical data HDD stuffs, including daily, weekly, monthly, and merging/updating.
    === July W2: ===
    :*Begin work on DBus interface for historical data. No IntraDayHistory support yet.
    === July W3: ===
    :*Finish DBus interface for historical data, including time conversion and adjusted prices.
    :*Finish/polish Previous-Close convenience function.
    === July W4: ===
    :*Implement Historical Data and Previous-Close test cases.
    :*Historical Data performance profiling and optimization.
    === August W1: ===
    :*Add DBus WatchList support.
    :*Watchlist test cases.
    :*Polish CompanyInfo/Stats DBus interface, add support for querying, key-name lookup, etc.
    === August W2: ===
    :*Saving/loading of CompanyInfo/Stats to disk.
    :*Improve/update CompanyInfo test cases.
    === August W3: ===
    :*Polish documentation, test cases, and further profiling.
    === August W4: ===
    :*Buffer
     
    === Time Permitting: ===
    :*Expand CompanyInfo by parsing data from Yahoo's Key Statistics page.
    :*Improved intraday real-time support by parsing data from finance.yahoo.com (or other source?)
    :*User-controlled CSV export.
    :*Add/finish support for IntraDayHistory.
    :*IntraDayHistory test cases.
     
    [http://techbase.kde.org/User:Bpeller# [Jump to Top]]
     
    == Planned Components ==
    *'''Always-On/On-Demand Backend:''' A member of [[Projects/KdeFinance/Alkimia|Alkimia]]
    *'''Always-On/On-Demand Backend:''' A member of [[Projects/KdeFinance/Alkimia|Alkimia]]
    ::''The backend handles all quote retrieval/storage, and provides a public interface for client applications to query over DBus for whatever information they need. The backend also serves as a centralized place for consolidating various stock-price-manipulation maths common to financial applications' needs.''
    ::''The backend handles all quote retrieval/storage, and provides a public interface for client applications to query over DBus for whatever information they need. The backend also serves as a centralized place for consolidating various stock-price-manipulation maths common to financial applications' needs.''
    Line 32: Line 84:




    == Backend Functionality ==
    == Planned Backend Functionality ==


    === '''Internals''' ===
    === '''Internals''' ===
    Line 65: Line 117:


    ===== '''Watchlist Management''' =====
    ===== '''Watchlist Management''' =====
    :*''Merge Watchlists''
    :*''Get Watchlists''
    :::'''Inputs:'''
    :::'''Output:'''
    ::::*List of all watchlist names
     
    :*''Get Watchlist Symbols''
    :::'''Inputs:'''
    ::::*Watchlist name
    :::'''Output:'''
    ::::*List of the watchlist's symbols
     
    :*''Add/Remove Watchlists''
    :::'''Inputs:'''
    ::::*Watchlist
    ::::*Action: Add or Remove
    :::'''Output:'''
    ::::*Bool Success
     
    :*''Lower Priority: Merge Watchlists''
    :::'''Inputs:'''
    :::'''Inputs:'''
    ::::*First Watchlist
    ::::*First Watchlist
    Line 72: Line 142:
    ::::::Merge: First into second, delete first. [Mirroring 'mv' command.]
    ::::::Merge: First into second, delete first. [Mirroring 'mv' command.]
    ::::::New: Create a new watchlist, preserve originals.
    ::::::New: Create a new watchlist, preserve originals.
    ::::*''Optional:'' New Watchlist's Name
    ::::::*''If Reqd:'' New Watchlist's Name
    :::'''Output:'''
    ::::*Bool Success
     
    :*''Add/Remove Watchlists''
    :::'''Inputs:'''
    ::::*Watchlist
    ::::*Action: Add or Delete
    :::'''Output:'''
    :::'''Output:'''
    ::::*Bool Success
    ::::*Bool Success
    Line 87: Line 150:


    ==== '''Intra-Day/Most-Recent-Close Quote''' ====
    ==== '''Intra-Day/Most-Recent-Close Quote''' ====
    :''This is a primary function.''
    :''This is a convenience function.''
    :::'''Inputs:'''
    :::'''Inputs:'''
    ::::*Symbol
    ::::*Symbol
    Line 134: Line 197:
    :::::::::*Or directly as a number of bars.
    :::::::::*Or directly as a number of bars.
    ::::*''Optional:'' Time Period: ''Specify by providing'':
    ::::*''Optional:'' Time Period: ''Specify by providing'':
    :::::::''Optional:'' A major time unit. (Minutely, Daily ''(Default)'', Weekly, or Monthly)  
    :::::::''Optional:'' A major time unit. (Daily ''(Default)'', Weekly, or Monthly)  
    :::::::''Optional:'' A multiplier. ''Default = 1''
    :::::::''Optional:'' A multiplier. ''Default = 1''
    ::::*''Optional:'' Adjusted or Absolute Prices
    ::::*''Optional:'' Adjusted or Absolute Prices
    Line 156: Line 219:
    #Support for storing/browsing/querying sectors and industries, as well as tracking their performance using (1) and (2).  
    #Support for storing/browsing/querying sectors and industries, as well as tracking their performance using (1) and (2).  
    #Improved support for options, futures, and FOREX.
    #Improved support for options, futures, and FOREX.
    #[Maybe] Technical Analysis Indicator Calculation Support: Essentially this would be providing a Qt-like API wrapped around [http://ta-lib.org/index.html TA-Lib]'s [http://ta-lib.org/d_api/d_api.html#Abstraction Abstract Interface]. Interface documentation [http://ta-lib.svn.sourceforge.net/viewvc/ta-lib/trunk/ta-lib/c/include/ta_abstract.h?view=markup here]. ''If this were to be done, it would be as an optional plugin, detected at runtime.''  
    #[Maybe] Technical Analysis Indicator Support: Essentially this would be providing a Qt-like API wrapped around [http://ta-lib.org/index.html TA-Lib]'s [http://ta-lib.org/d_api/d_api.html#Abstraction Abstract Interface]. Interface documentation [http://ta-lib.svn.sourceforge.net/viewvc/ta-lib/trunk/ta-lib/c/include/ta_abstract.h?view=markup here]. ''If this were to be done, it would be as an optional plugin, detected at runtime.''  
    #[Depends on TA-Lib] Manual calculation of Beta with controllable inputs for the time period and the "volatility base market." ''This ability is probably most applicable for large private investors, portfolio analysts, and institutions/banks, and is particularly relevant for those invested in international/emerging markets who do not want to base their volatility calculations off of American markets, as Yahoo's provided Beta [http://www.investopedia.com/articles/financial-theory/09/calculating-beta.asp is assumed] to do.''
    #[Depends on TA-Lib] Manual calculation of Beta with controllable inputs for the time period and the "volatility base market." ''This ability is probably most applicable for large private investors, portfolio analysts, and institutions/banks, and is particularly relevant for those invested in international/emerging markets who do not want to base their volatility calculations off of American markets, as Yahoo's provided Beta [http://www.investopedia.com/articles/financial-theory/09/calculating-beta.asp is assumed] to do.''


    [http://techbase.kde.org/User:Bpeller# [Jump to Top]]
    [http://techbase.kde.org/User:Bpeller# [Jump to Top]]

    Latest revision as of 08:36, 28 May 2010

    Central Stock Price Information

    A proposal for Season of KDE 2010 by Brian Cappello
    Mentors: Cristian Onet and Alvaro Soliverez

    Use Cases

    • A plasmoid wants to display some stock quotes stored in a watchlist.
    • Santosh Kumar wants to provide notification support for changes in stock prices.
    • An investor wants to chart an investment's yearly performance starting from their first purchase thirty-five years ago.
    • An investor wants to do some fundamental research on a stock they're considering trading.
    • A day trader wants to chart today's 5-minute price action since the open.
    • A swing trader wants to chart two months worth of hourly data.
    • Another swing trader wants to chart 200 bars of daily data.
    • A student of the markets wants to "replay" the crash of '87 starting from 3 months prior and advancing one day at a time.
    • A mechanical systems trader writes a scripted trading algorithm, and they want to back-test its performance on various sets of historical data.

    [Jump to Top]


    Tentative Schedule: WIP/subject to change

    May W4:

    • Continue research into DBus, asynchronous transfers, doctools, unit tests and cmake.
    • Get some DBus code working!

    June W1:

    • Attempt to "finalize" class/object separation and interaction planning.
    • Preliminary DBus symbol management. Add/remove only, no watchlists yet.
    • Implement Yahoo historical data interpreter and runtime-only storage. (Concentrate on daily data. W/M/IntraDayHistory later.)

    June W2:

    • Implement Yahoo intraday quotes support and runtime merging/storage into most-recent-index of daily historical data.
    • Get on-demand DBus intraday support working.
    • Figure out how best to store data to HDD. Possibly as flat-file CSVs and/or KConfig?
    • Get SymbolManager storage-to/from-disk working.

    June W3:

    • Get intraday quotes monitoring working. Broadcast notification of updates to clients.
    • Implement IntraDay quotes test cases.
    • Get preliminary on-demand DBus CompanyInfo/Stats working (assume client knows what keys are available).

    June W4:

    • CompanyInfo Test cases.
    • Begin to get HDD saving/loading historical data working (maybe merging/updating).

    July W1:

    • Finish up historical data HDD stuffs, including daily, weekly, monthly, and merging/updating.

    July W2:

    • Begin work on DBus interface for historical data. No IntraDayHistory support yet.

    July W3:

    • Finish DBus interface for historical data, including time conversion and adjusted prices.
    • Finish/polish Previous-Close convenience function.

    July W4:

    • Implement Historical Data and Previous-Close test cases.
    • Historical Data performance profiling and optimization.

    August W1:

    • Add DBus WatchList support.
    • Watchlist test cases.
    • Polish CompanyInfo/Stats DBus interface, add support for querying, key-name lookup, etc.

    August W2:

    • Saving/loading of CompanyInfo/Stats to disk.
    • Improve/update CompanyInfo test cases.

    August W3:

    • Polish documentation, test cases, and further profiling.

    August W4:

    • Buffer

    Time Permitting:

    • Expand CompanyInfo by parsing data from Yahoo's Key Statistics page.
    • Improved intraday real-time support by parsing data from finance.yahoo.com (or other source?)
    • User-controlled CSV export.
    • Add/finish support for IntraDayHistory.
    • IntraDayHistory test cases.

    [Jump to Top]

    Planned Components

    • Always-On/On-Demand Backend: A member of Alkimia
    The backend handles all quote retrieval/storage, and provides a public interface for client applications to query over DBus for whatever information they need. The backend also serves as a centralized place for consolidating various stock-price-manipulation maths common to financial applications' needs.
    • A "Master" Backend UI: Some form/combination of plasmoid/application/kpart/kcm...
    Cumulative functionality should support something along the lines of:
    • Managing which symbols and watchlists the backend permanently stores data for.
    • Configuring the backend's default historical and intra-day data sources. (And maybe resource/performance settings.)
    YahooCSV: Download quotes from Yahoo! (Default for Intra-Day & Historical)
    Directory: Import/monitor quotes from a directory containing CSV files. (For users with paid-data services.)
    Time Permitting: YahooWEB: Parse real-time data from the finance.yahoo.com website. (Intra-Day only)
    • Exporting user specified data/information/statistics to CSV.

    [Jump to Top]


    Planned Backend Functionality

    Internals

    • Maintain/store a master list of all watchlists.
    • Maintain/store a master list of all symbols to be watched. Filters out watchlist-duplicates and includes symbols which aren't a member of any watchlist.
    • Maintain a local/offline store of downloaded data:
    1. Company info and statistics. (Used for offline mode/throughout the day. Company stats change frequently enough and are reasonably cheap (fast) to download so we update this once daily, if possible.)
    2. Accumulate sampled Intra-Day data as Minutely data.
    3. Store and keep up-to-date "primary" data: Minutely, Daily, Weekly, and Monthly Time Periods

    [Jump to Top]


    Public DBus Interfaces

    Management of stored symbols and watchlists

    These are primary functions.

    Symbol Management
    • Add/Remove Symbols
    Inputs:
    • Symbol
    • Action: Add or Remove
    Output:
    • Bool Success
    • Modify a watchlist's symbols
    Inputs:
    • Symbol
    • Watchlist
    • Action: Add or Remove
    Output:
    • Bool Success
    Watchlist Management
    • Get Watchlists
    Inputs:
    Output:
    • List of all watchlist names
    • Get Watchlist Symbols
    Inputs:
    • Watchlist name
    Output:
    • List of the watchlist's symbols
    • Add/Remove Watchlists
    Inputs:
    • Watchlist
    • Action: Add or Remove
    Output:
    • Bool Success
    • Lower Priority: Merge Watchlists
    Inputs:
    • First Watchlist
    • Second Watchlist
    • Action: Merge or New
    Merge: First into second, delete first. [Mirroring 'mv' command.]
    New: Create a new watchlist, preserve originals.
    • If Reqd: New Watchlist's Name
    Output:
    • Bool Success

    [Jump to Top]


    Intra-Day/Most-Recent-Close Quote

    This is a convenience function.
    Inputs:
    • Symbol
    • Optional: Prices or Percents
    Prices: the real prices (Default)
    Percents: Percent changes from previous close
    Output:
    • struct: Contains a KDateTime, the Open, High, Low, Close, Volume, and, if applicable, Open Interest (eg for futures). Optional: and Beta.

    Previous Day's Close Quote

    This is a convenience function.
    Inputs:
    • Symbol
    • Optional: Prices or Percents
    Prices: the real prices (Default)
    Percents: Percent changes from previous close (eg two closes ago)
    Output:
    • struct: Contains a KDateTime, the Open, High, Low, Close, Volume, and, if applicable, Open Interest (eg for futures). Optional: and Beta.

    Company Information, Key Statistics and Fundamentals

    This is a primary function.
    Inputs:
    • Symbol
    • Parameters (keys)
    Initially most (all applicable) parameters easily downloadable from Yahoo will be supported. Time permitting, more information sources can be added (such as parsing parameter values from Yahoo's Key Statistics webpage).
    Time Permitting: There should probably be a way to query at runtime what keys are available, as well as key-lookup by human-readable names.
    Output: Must be able to accommodate both strings and numbers.
    • A container object holding the requested parameters' key-value pairs.

    [Jump to Top]


    Historical Quotes

    This is a primary function.
    Inputs:
    • Symbol
    • Amount/duration of historical data:
    Optional: Specify an end date. (Defaults to the most recent close.)
    Specify a duration (relative to the end date) by providing one of the following:
    • A start date.
    • An elapsed quantity and unit of time, independent of the (optionally) requested time period.
    • Or directly as a number of bars.
    • Optional: Time Period: Specify by providing:
    Optional: A major time unit. (Daily (Default), Weekly, or Monthly)
    Optional: A multiplier. Default = 1
    • Optional: Adjusted or Absolute Prices
    Adjusted: (Default) Prices adjusted for splits and dividends.
    Absolute: Raw price data. (Provided by default if the requested symbol has no splits or dividends.)
    • Optional: A maximum lookback. Default = 0
    The number of "hidden" bars (older than the start date) to return. Used by technical analysis indicators so their results will be complete for the requested length of "shown" price data.
    Output:
    • A container object holding the input parameters used to produce the data, as well as the data itself (containing the same information as Intra-Day structs). Need to look into what would be the most convenient storage method for MVC-based clients such as KDChart as used by KMyMoney, but that's still flexible enough to be non-MVC-friendly.

    [Jump to Top]


    Future Backend Possibilities

    Time Permitting / Low Priority:

    1. Support for calculating relative price/performance between multiple data sets.
    2. Support for calculating "custom Market Indexes" from watchlists.
    3. Support for storing/browsing/querying sectors and industries, as well as tracking their performance using (1) and (2).
    4. Improved support for options, futures, and FOREX.
    5. [Maybe] Technical Analysis Indicator Support: Essentially this would be providing a Qt-like API wrapped around TA-Lib's Abstract Interface. Interface documentation here. If this were to be done, it would be as an optional plugin, detected at runtime.
    6. [Depends on TA-Lib] Manual calculation of Beta with controllable inputs for the time period and the "volatility base market." This ability is probably most applicable for large private investors, portfolio analysts, and institutions/banks, and is particularly relevant for those invested in international/emerging markets who do not want to base their volatility calculations off of American markets, as Yahoo's provided Beta is assumed to do.

    [Jump to Top]