Welcome to the Forum

Creating an account is currently only possible via registration at SimFin.

Limit to the number of values returned from a search

I built a JSON to request all of the indicatorID's that the list says are available (there are 77 total). This JSON only requests for one ticker for 4 time periods (ttm, ttm-1, ttm-2, ttm-3). This means that it's asking for data for 308 values every time it runs (4*77). Every time I run it, it returns results for all of the requests but everything after the 100th request is null. I only get the first 25 metrics that I requested (some of those are null also presumably since they don't have any value in the database).

Is this because everything after the first 25 metrics on the list actually isn't available? Or is it because there is a limit to the amount of indicatorID's you can put in your request? The "100" number seems too exact for it just to be that the rest aren't yet available but who knows?

Comments

  • After further investigation, the missing data seems to exist but it's pretty random what data will be returned as true data and what data will just be filled in with a "null" value.

    If I request 77 indicators, it returns only the first 25. If I request 25 indicators, it returns 2, 3, and 11 to 25. If I request 1 indicator it returns it every time.

    Seems like it should be a coding error but I've been over it many times.

    I'm thinking now that this question under the wrong discussion topic though as the data is there.
  • edited February 22
    Hi,

    ok weird, I assume you are using the data finder API? can you maybe post the JSON you are sending to the API?
  • Yes this is the data finder API that I'm using. This example of the JSON shows 2 indicator ID values where as what I've been doing is adding up to 77. Obviously I've used arrays throughout this whole JSON so that I don't have to run through and change 300 or so values by hand if I ever need to. The other wild card here is that I'm programming this with Google Script Editor and dumping all the outputs into a Google Sheets spreadsheet. I haven't seen any examples of anyone doing specifically what I'm doing yet so maybe there's an issue there.

    var TimeValue = ["ttm","ttm-1","ttm-2","ttm-3"];
    var Operator = ["eq","eq","eq","eq"];
    var MetaID = [6,6,6,6];

    var Data03 =
    {
    "search":
    [
    //GRAB 4 YEARS OF THIS INDICATORID
    {
    "indicatorId": IndicatorID[0],
    "meta": [{"id": MetaID[0], "value": TimeValue[0], "operator": Operator[0]}]
    },
    {
    "indicatorId": IndicatorID[0],
    "meta": [{"id": MetaID[1], "value": TimeValue[1], "operator": Operator[1]}]
    },
    {
    "indicatorId": IndicatorID[0],
    "meta": [{"id": MetaID[2], "value": TimeValue[2], "operator": Operator[2]}]
    },
    {
    "indicatorId": IndicatorID[0],
    "meta": [{"id": MetaID[3], "value": TimeValue[3], "operator": Operator[3]}]
    },
    //GRAB 4 YEARS OF THIS INDICATORID
    {
    "indicatorId": IndicatorID[1],
    "meta": [{"id": MetaID[0], "value": TimeValue[0], "operator": Operator[0]}]
    },
    {
    "indicatorId": IndicatorID[1],
    "meta": [{"id": MetaID[1], "value": TimeValue[1], "operator": Operator[1]}]
    },
    {
    "indicatorId": IndicatorID[1],
    "meta": [{"id": MetaID[2], "value": TimeValue[2], "operator": Operator[2]}]
    },
    {
    "indicatorId": IndicatorID[1],
    "meta": [{"id": MetaID[3], "value": TimeValue[3], "operator": Operator[3]}]
    },
    ],

    "simIdList": [TickerID]
    }
  • thanks. I have some other stuff to do currently but I'll check it over the weekend.
  • edited March 4
    ok tested it now, when I translate your code to Python and try it for Apple it works fine (checked the results from the API with the actual values and they are identical). So that is my code (Python):
    
    TimeValue = ["ttm","ttm-1","ttm-2","ttm-3"]
    Operator = ["eq","eq","eq","eq"]
    MetaID = [6,6,6,6]
    IndicatorID = ['1-1','1-2']
    
    
    
    Data03 = {
    "search":
    [
    {
    "indicatorId": IndicatorID[0],
    "meta": [{"id": MetaID[0], "value": TimeValue[0], "operator": Operator[0]}]
    },
    {
    "indicatorId": IndicatorID[0],
    "meta": [{"id": MetaID[1], "value": TimeValue[1], "operator": Operator[1]}]
    },
    {
    "indicatorId": IndicatorID[0],
    "meta": [{"id": MetaID[2], "value": TimeValue[2], "operator": Operator[2]}]
    },
    {
    "indicatorId": IndicatorID[0],
    "meta": [{"id": MetaID[3], "value": TimeValue[3], "operator": Operator[3]}]
    },
    {
    "indicatorId": IndicatorID[1],
    "meta": [{"id": MetaID[0], "value": TimeValue[0], "operator": Operator[0]}]
    },
    {
    "indicatorId": IndicatorID[1],
    "meta": [{"id": MetaID[1], "value": TimeValue[1], "operator": Operator[1]}]
    },
    {
    "indicatorId": IndicatorID[1],
    "meta": [{"id": MetaID[2], "value": TimeValue[2], "operator": Operator[2]}]
    },
    {
    "indicatorId": IndicatorID[1],
    "meta": [{"id": MetaID[3], "value": TimeValue[3], "operator": Operator[3]}]
    },
    ],
    "simIdList": [111052]
    }
    
    And those were the results:
    {"totalPages":1,"currentPage":1,"firstPos":1,"lastPos":1,"totalItems":1,"results":[{"simId":111052,"name":"APPLE INC","values":[
    {"indicatorId":"1-1","value":261612000000},
    {"indicatorId":"1-1","value":239176000000},
    {"indicatorId":"1-1","value":218118000000},
    {"indicatorId":"1-1","value":234988000000},
    {"indicatorId":"1-2","value":161654000000},
    {"indicatorId":"1-2","value":147254000000},
    {"indicatorId":"1-2","value":134102000000},
    {"indicatorId":"1-2","value":140680000000}]}]}
  • Yep thanks for checking on that for me. It does appear that smaller amounts of data being requested do return correct information. It's only when I query for larger amounts of data that some seems to get lost. I'm trying to determine if it's my code or if it's connected to using Google Script Editor with no Python interface.

    I attached a more full example that mimics what you did with more data requested. Would you mind seeing if this works? I figure there will be some "null" values tucked into different places that may not have data but I expected more major things like revenue to at least appear.

  • sorry I didnt't have time to test this yet, I know for certain that the data finder API should be working well with something like 60 indicators for single companies (for the current TTM time periods), since the "key indicators" info in the company profiles is coming from there. I'll see if I can test your code next weekend.
  • we have to limit the number of requested fields from the finder now anyway since it's hard to handle for the server otherwise, so I'll close this
This discussion has been closed.