Dealing with price spikes

[bug-or-feature]

I'm interested to hear how other people deal with price spikes. I seem to spend way too much time dealing with them

• What resources do you use to check prices?
• Do you have custom values set for max_price_spike? If so, what value?

[robcarver17]

One issue is that the system is currently set up to allow zero volume prices which I think results in more price spikes. I'm going to make this configurable.

[bug-or-feature]

Sounds good. But say you get a price spike, what do you do to work out whether to accept the IB price, or replace it with some other price? I think I remember reading somewhere that you thought that IB prices could be 'flaky' (though I can't now find where I read that), so what do you check it against? Barchart, somewhere else?

[robcarver17]

I mostly get false positives on infrequently sampled data. The way the spike checker works, is that it assumes that prices follow a normal distribution in sample time, which means if you have something that misses a few days of data then it gets flagged as a spike even if it isn't. This usually happens in illiquid stuff, or in contracts I'm not currently pricing off (which means I don't really care, unless I do something dumb like trade an RV system off them).

If something has moved so much as to genuinely be a spike, it's normally in the news.

It's very rare to have a true error. The only true spike errors I've had have been obvious errors: zero prices, or -1.

[bug-or-feature]

So it sound like what we need is a value for max_price_spike that flags zero or -1 prices, and ignores everything else. Anyone know what that value is?

I looked at trying to print out what the price spike value was, when you run interactive_manual_check_historical_prices(), but it wasn't trivial to do

The only RVs I know about are the kind you drive out into the desert to cook meth - I suspect you mean something else?

[robcarver17]

Not currently configurable through changing max_price_spike. I'm going to change the price spike thing so you pass in a function name that does the cleaning (with parameters you can change). It's on the issues list to do. Better than having magic numbers for the spike value.

RV = relative value.

[meldinman]

The only bad prices I seem to regularly get are for the ethereum and micro etherum contracts. Not sure why IB seems to sometimes get bad quotes for those...

[bug-or-feature]

Not currently configurable through changing max_price_spike. I'm going to change the price spike thing so you pass in a function name that does the cleaning (with parameters you can change). It's on the issues list to do. Better than having magic numbers for the spike value.

Thanks @robcarver17 look forward to it

[craigmediaservices]

@bug-or-feature I just encountered this. It appears the current solution is to simply set max_price_spike at a high value such as 20? I've spent a lot of time manually checking and the flags I'm getting appear to be valid price data. I'm running a new paper system that is only about a week old.

[bug-or-feature]

no change really, except Rob implemented the cleaning function thing

[craigmediaservices]

Thank you, I believe this is related:

Spike checker

max_price_spike: 8.0

Default get_GMT_offset_hours

GMT_offset_hours: 0

Behaviour of price filtering

ignore_future_prices: True
ignore_prices_with_zero_volumes_intraday: True
ignore_prices_with_zero_volumes_daily: False
ignore_zero_prices: True
ignore_negative_prices: False

The defaults appear to be valid, but I'm still getting a lot of flags on intraday prices (looks like 8.1, or 10, or sometimes 12) mostly on low volume - sometimes on a shift from very low to high. All the data appears to be valid, and I don't think I'd want to try to manually apply smoothing. Thanks though.