Hacker News

nkaz001
Show HN: High-frequency trading and market-making backtesting tool with examples github.com

apsears3 months ago

Where can I find (programming) tools for slow investing?

bequanna3 months ago

Options wheel strategy implementation from the ‘Theta Gang’ on Reddit: https://github.com/brndnmtthws/thetagang?tab=readme-ov-file

SkyMarshal3 months ago

I skimmed through the linked github and reddit post, and maybe I missed it, but this looks like the kind of strategy that gets blown up when a negative black swan occurs. They're "harvesting premium" by selling options, based on the fact that "implied volatility is typically higher than realized volatility on average." Which is true, until it isn't. Like, yikes dude.

For example, the author recommends spreading selling options across multiple sectors to avoid getting blown up by any one sector moving against you, implicitly assuming no correlation across sectors. But how many times do we have to relearn that lesson that correlations do happen, and more frequently than we expect.

One of the commenters in the reddit thread even says that - they retired just before 2020 b/c all their Monte Carlo simulations showed they would have plenty to retire on, unless a highly unlikely correlated whole-market crash occurred. Which happened in the same year after they retired, blowing up their retirement planning.

I wouldn't recommend this strategy to noobs.

coin3 months ago

Absolutely. I call it the options fallacy. It gives the illusion of risk free money but in reality it’s just capitalizing on a particular market pattern. There’s no free lunch.

someguydave3 months ago

Yeah selling options makes you an insurance firm, so you should have a better handle on valuing risk than your counterparty or you are probably the fool.

thorncorona3 months ago

optionsellers.com moment lol

almostgotcaught3 months ago

> Which is true, until it isn't.

this is the worst newspeak/vernacular/neologism of the whole bunch. it conveys/express literally nothing.

isoprophlex3 months ago

It conveys to me that black swans happen, and best to take them into account. I don't get your point.

anonymoushn3 months ago

This strategy has no alpha, best to ignore.

zer0tonin3 months ago

Literally negative alpha because you're losing upside with the covered call

sciencesama3 months ago

the concept is if i run this today how much will i make in a week !! we need a place to track that if we can !!

asavinov3 months ago

You could check out a tool for trade signal generation based on machine learning and feature engineering:

https://github.com/asavinov/intelligent-trading-bot

It trains ML models based on historic data and custom features and then uses them to generate a kind of intelligent indicator between -1 and +1. This intelligent indicator is then used to make trade decisions. Frequency is a parameter and can vary from 1 minute for crypto trading to 1 day for normal exchanges.

wslh3 months ago

How do you define slow investing? Mean something like fundamental analysis? A combination of fundamental analysis with riskier bets (e.g. crypto)?

randomcarbloke3 months ago

there are many more examples of slower/shorter term strategies than merely fundamental, there always have been...especially in the institutional space where the race to zero is functionally over.

bormaj3 months ago

Are you a professional in the field or is this your side project?

The quantitative trading posts on here typically just scratch the surface, but I have to say that I'm impressed with this one. Thanks for sharing!

nkaz001op3 months ago

No, that's why the project focuses more on cryptocurrency, which offers easier access to data and the market. I'm trying to understand trad-fi markets and implementing L3 market-by-order backtesting using Databento data, but creating a live bot without the market access would be impossible.

RayVR3 months ago

Interactive brokers is probably the best non-professional setup I’m aware of. You won’t get hft level latencies or price feeds.

to get good historical data in “trad-fi” you would need to map all the trades by code to filter out bad ticks before aggregating into bars. This alone is enough of a barrier to keep most retail traders out of the space

nkaz001op3 months ago

databento provides high-quality market-by-order data at a reasonable price, so at least I can play with the data, though I'm unsure about actual trading. That's why people including me like crypto.

RayVR3 months ago

from what I understand, databento is just aggregating exchange data. They must have some deal for processed data. Their MBO data is not cheap and they don't have much history. I would not trust their bars without a whitepaper documenting how they clean each market.

someguydave3 months ago

“by code” what codes? Do you mean offered but cancelled bids/asks and things like that?

RayVR3 months ago

No. The major concern is that bad ticks make it through your filters, which is more than likely going to make your models do something wonky.

It's been many years since I worked on cleaning data but essentially each event you receive is mapped to a code. Depending on your data provider, those codes may change at any time. Thomson Reuters is/was quite notorious for this. Maybe after Refinitiv spun out they cleaned up that mess but from what I have heard it is unlikely.

wslh3 months ago

As far as I know, HFT involves having close and specific network links to the markets, and you cannot beat the big players in that field. Am I missing something?

I have a story to tell, anonymizing the actors: I had an appointment to have dinner with a friend who works in cybersecurity at a top hedge fund in NY. Before we entered the restaurant, he received a call because a link with Hong Kong was failing. I told him that I assumed our dinner would be canceled, but somehow the issue was solved in 10 minutes. In this context, I understand that if HFT could be done using any Internet connection, there would not have been any issue there.

noitpmeder3 months ago

I see this is tailored toward crypto exchanges -- what, if any, are the major differences between crypto based trading and more traditional stock/future/option exchanges?

Prunkton3 months ago

to give some examples:

- you can trade crypto 24/7, this sounds obvious but on the other hand side this also means, there is no pre-market trading and all the obscure things attacked to it

- before blocks are being created, transactions are usually collected in so called mempools. (Way oversimplified) block proposers or miners, who are selected to create the next block, can choose which transactions to include, as space is limited. They can also determine the order of these transactions. All of this is publicly visible and opens a lot of opportunities to harvest slippage etc. (lookup MEV-bots)

- generally speaking, there is no robin hood or other intermediary, that can block you from trading

- in crypto, you essentially have access to all available financial products without any barriers to entry compared to traditional finance

it looks like this repository is using Binance APIs for trading. So my statements are no entirely true for this case. But you can use trading bots like this on decentralized exchanges or DeFi products like curve finance without being dependent of an intermediary

derf_3 months ago

> All of this is publicly visible and opens a lot of opportunities to harvest slippage etc. (lookup MEV-bots)

Basically, people can front-run your trades, and this is built into the market by design. Also, instead of the winner being the fastest like in tradfi, there is a competitive auction where participants pay for priority. See, e.g., https://archive.ph/W0nvi or pages 7...9 of https://assets.ey.com/content/dam/ey-sites/ey-com/en_us/topi... for examples. Even if you are not the one doing the front-running, you have to be aware that someone else will be.

To make it more exciting there are also implementation bugs: https://archive.ph/9w32t

lern_too_spel3 months ago

HFT on chain is a funny concept. 1.5 milliHertz, and the trading fees would wipe you out.

PaulRobinson3 months ago

I'm not into crypto, but I imagine it is possible to trade on exchanges with regards to price movements without trading on the chain directly. This is either a good thing (higher volumes at higher transaction speeds), or creates the exact thing crypto is meant to avoid (see: FTX).

ta126534213 months ago

Yes, you can trade crypto 24/7 - in theory, because liquidity is muuuch lower outside the regular daily hours

In crypto, you have only two assets - BTC and ETH, all other assets are not tradeable because of volume/liquidity, sure you can & sell them, but most trading strategies wont work well in underliquid markets

robxorb3 months ago

This seems off. Sufficient liquidity in a market isn't natively a binary state. That state would be imposed by the programmer with a threshold. Hardcoding only two assets would surely miss dynamic opportunities in others.

lern_too_spel3 months ago

The crypto exchange is likely underregulated and frontrunning you, and since its assets are kept on a blockchain where its wallet can be drained with no recourse instead of in a ledger governed by laws, you are more likely to have your money simply disappear and reappear in a North Korean wallet. That is the most significant difference. Otherwise, it's just like any other market.

nurettin3 months ago

Crypto trading has no regulations, huge spreads and illogical commissions. Your online"wallet" doesn't contain the actual asset, just numbers. If the host goes bankrupt, gets hacked or just turns evil, you have no proof of ownership to sue.

The closest thing to legit crpto trading is to trade CME's btc/eth futures, but they don't have much volume or data to backtest with.

miohtama3 months ago

First of all in traditional stock/futures exchanges you cannot connect to the exchange itself, but must go through a broker due to the regulation. This means you cannot directly interact with the order book, needed for the high frequency trading.

Cryptocurrency exchanges are direct-to-retail so anyone can run HFT strategies and act as a market maker. Thus, cryptocurrency trading is “more democratized.”

Bluecobra3 months ago

You can absolutely host your networking gear/servers inside a co-location facility where the underlying exchange is hosted. It just requires tons of money to do so.

gottorf3 months ago

> It just requires tons of money to do so.

On the order of thousands of dollars a month[0], depending on how close you want to get.

[0]: https://www.cmegroup.com/globex/connectivity-options.html

djtango3 months ago

Realistically that isn't that much money unless you want to be some kind of hobbyist HFT tinkerer.

With no stance on whether people should be doing HFT as a hobby, presumably there is a finite amount of colocation space available and an opportunity cost/maintenance associated with onboarding people so this is what the market and regulators decided were the table stakes?

gottorf3 months ago

Yes, it's really not that much money, all things considered. Even regular colocation (just for hosting) at a name-brand facility with all the redundancy that the CME facility has would run you $1-2k a month.

nkaz001op3 months ago

Sadly, it is becoming more like trad-fi. In the past, anyone could receive rebates for market making, but these days, only high-tier traders, trading companies, can benefit from them. The tier hurdle is getting higher. Also, there are now different APIs and limits for individual traders and high-tier trading companies.

TeaBrain3 months ago

Brokers exist as they are providing the service of managing and routing orders to an exchange, which market participants in electronic exchanges wouldn't be able to interact with otherwise, without having a colocation spot on the exchange and knowing FIX protocol. Retail traders don't know, don't want to learn, and really have no reason to learn how to route unmanaged orders via FIX protocol. If you want to directly interact with an exchange you can do so, you'll just have to buy or lease a colocation spot on an exchange first.

Cryptocurrency exchanges are like many FX exchanges. They are not exchanges like the regulated stock, futures and options exchanges are, but are more like decentralized trading pools with unsynchronized order books, which are opaquely operated by a single broker. In crypto exchanges, it is not that there is no broker, but that the exchange and the broker are the same entity. There is no standardized protocol to route independently managed orders directly to crypto exchange servers. There's usually no way to directly interact with the exchange server at all.

cturner3 months ago

A major difference is the clearing model. The asset classes you mention generally have a clearing house behind them. This is designed to protect the system against counterparty risk. If one party goes under, it has limited contagion, even to the people who were (in practical terms) on the other side of the trades.

This layer of protection allows better specialisation. In crypto, you need to trade on your own balance sheet. In centrally cleared markets, it is routine for trading firms to lease balance sheet from banks, who have lots of capital and good risk management at scale, but who are typically less effective at trading specific markets competitively. This leads to more liquidity being available and more competitive markets.

RayVR3 months ago

Banks aren’t trading because post-2008 regulations made it nearly impossible. They have plenty of money to hire top trading talent and did so for a long time.

[deleted]3 months agocollapsed

mhh__3 months ago

The data for traditional finance is extremely expensive.

koalala3 months ago

i've been looking into doing this, the basic algorithms you can get from research, but building a top of the line FPGA design for this is beyond my capabilities as a single engineer, does anyone know of anyone doing this with just some really well optimized c++?

vazma3 months ago

You can't compete with HFT firms even if you have this equipment because you need to install your equipment in a colocation, aka a data center that exchanges sending their raw data and your equipment trades there. In this kind of trading receiving the data nanoseconds late from the other players matters.

lend0003 months ago

This is only true for arb opportunities on exchanges that offer colocation (which leaves almost all crypto exchanges as fair game). But if you can't get colocation, there isn't much point in having the FPGA anyway since the network latency is going to be so much higher than any savings from FPGA vs low level programming.

koalala3 months ago

yes, this would be non-colocated on AWS. wouldn't everyone have the same network latency if they had a node in the same AWS region? and wouldn't that even the playing field, making FPGAs relevant again (you can rent an FPGA there)? i guess the importance of low latency also depends on the update rate of the exchange server.

robbyiq9993 months ago

Is all of this just a fancy complex way to lose money? Isn’t impossible to beat the market consistently?

3923 months ago

Yep, that's why there are no more prop trading shops.

TacticalCoder3 months ago

> Show HN: High-Frequency Trading... > currently for Binance Futures and Bybit

There is no high-frequency trading in the cryptocurrencies world. It's medium frequency trading at best.

These cryptocurrencies exchanges (really broker+exchange mixed as one) aren't serving the data feed anywhere near quickly enough nor executing orders quickly enough to approach HFT.

Firm doing HFT are co-locating near the exchange and using direct data feeds, at times using algorithms running on FPGA. Stuff like that.

That's HFT.

What's shown for Binance/Bybit is simply not HFT.

nkaz001op3 months ago

I partly agree with you. Firstly, this is primarily a backtesting tool, allowing you to test under different latency setups. The example in the project demonstrates its use. Secondly, crypto exchanges is slow. Even though they provide low-latency APIs to higher-tiered traders, but I believe they are still slower compared to trad-fi. But, the market data is public, there might be differences in the data received by higher-tiered traders. These enable individuals to analyze the market and start projects like this.

energy1233 months ago

As long as orders are processed in a FIFO fashion, then you can have HFT type strategies, regardless of whether it takes one microsecond or one year to process those orders.

[deleted]3 months agocollapsed

bottom999mottob3 months ago

So how is the latency floor for HFT calculated?

Bluecobra3 months ago

Order fill ratio. Whoever has the fastest hardware/software wins.

kasey_junk3 months ago

That’s not really true in traditional HFT because resting orders no matter how slowly placed will take precedence over orders that are new.

Typically you’d have some combination of order fill for orders that cross the book, successful cancellation (which is where the real race is) and mechanical observation of tick to trade.

650nanos3 months ago

> That’s not really true in traditional HFT because resting orders no matter how slowly placed will take precedence over orders that are new.

No. Not everything is FIFO.

You can be last to place a passive resting/limit order at a price and still be the first to receive a fill at that price. And yes, you had the advantage of seeing all the other passive orders at that price.

Examples: Pro-rata markets (SOFR interest rates, some US treasuries), designated market makers (futures, equities), etc.

Source: I work in HFT.

nkaz001op3 months ago

Is designated market maker able to get filled before an order in the FIFO order queue at NYSE, NASDAQ or CME?

ickyforce3 months ago

CME documentation states:

"LMM – CME Designated Lead Market Makers are each allocated a configurable percentage of an aggressor order quantity before the remaining quantity passes to the next step."

Certain matching algorithms allocate orders to LLM before considering FIFO - specifically algorithm T (LMM w/o Top). So I guess it depends on how strict is your definition of "FIFO order queue".

https://cmegroupclientsite.atlassian.net/wiki/spaces/EPICSAN...

https://cmegroupclientsite.atlassian.net/wiki/spaces/EPICSAN...

(I don't know the internals of the mentioned exchanges)

[deleted]3 months agocollapsed

650nanos3 months ago

> Is designated market maker able to get filled before an order in the FIFO order queue at NYSE, NASDAQ or CME?

“Unfortunately, I cannot comment at this time.”

WorkerBee284743 months ago

I think on-the-floor traders actually do. I can't find the source to back that up, though.

fmap3 months ago

nkaz001op3 months ago

Thank you! I know that NASDAQ and CME also have designated market makers, but I couldn't find any specific privileges they have over non-market makers, apart from the different fee schedules. Is the ability of NYSE designated market makers to handle order fills a common practice across other exchanges?

[deleted]3 months agocollapsed

smabie3 months ago

lol no one thinks like that

Yes there is hft (just like how there was hft in tradfi 20 years ago when latency was measured in the ms)

cedws3 months ago

Yeah, it's awful. Used to work for a crypto market maker.

Binance run all their stuff on AWS behind CloudFront. If you want to make orders, you have to use a slow REST API via their CDN. Lots of jitter, and spikes. Their tech is probably not very well engineered for latency.

[deleted]3 months agocollapsed

yobid203 months ago

It's in rust. Somebody port it to C++ then it might be worth looking at.

stargrazer3 months ago

https://github.com/rburkholder/trade-frame/blob/master/lib/T...

Part of a simulator handling limit orders and market orders. I run this real time against a DTN IQfeed live market feed to perform some testing for options trading/execution.

ta126534213 months ago

I bet 100 USD that your tool is not working :-)

Why?

BACKtesting is not comparable to real market scenarios.

What you should do before going live, is a so call Walk-Forward-Test - if this version shows your profits, then launch.

RayVR3 months ago

this is not a trading strategy.

This is a tool that supports simulated execution for strategies. The strategies are left for users of the tool to implement.

I have no idea if this code works or if the features listed are finished but if done well it’s a useful baseline for anyone interested in algorithmic trading.

nurettin3 months ago

Walk-forward is just back testing with out of sample periods where your parameters are constant. You can emulate it with a backtester by reducing parameter space to one in those periods.

ta126534213 months ago

OK,

i will give you 1000 USD if you deliver an algorithm that delivers similar returns for the same asset class / same timeframe with back-testing and second run with walk-forward-testing :-))

nurettin3 months ago

I don't think you understand the explanation. It isn't the same timeframe.

Months 1-6 backtest and optimize.

Months 7-8 backtest, no optimization. Use best parameters from 1-6.

Months 3-8 backtest and optimize.

Months 9-11 backtest, no optimization. Use best parameters from 3-8.

Walk forward with only backtest.

ta126534213 months ago

Looks like we are talking about two different things.

What you describe is not a walk-forward-test which adds data continuously on a given frequency, like daily (or hour or whatever)

nurettin3 months ago

What you described sounds like just forward testing. You do this for a few weeks to make sure that live trading will match backtests.

See the docs for walk forward optimization here

https://help.tradestation.com/09_01/tswfo/topics/about_wfo.h...

ta126534213 months ago

Mh, interesting: i was told this is called "walk forward testing" - nontheless: thanks for clarification!

Exactly what you describe is what we are doing: Since in backtesting, our algo works more than perfect - if we switch to "live-data-scenario", meaning adding data bar by bar (per timeframe), the system behaves differently and the results are a little bit worse.

Therefore my sentence above: During intensive testing we found out that most of those strategies fail (nearly completely) in real-world-market-scenarios.

Tradestation: We have built our own app, we are aware of these "of-the-shelf" solutions, but they are fairly limited regarding what we are doing.

[deleted]3 months agocollapsed

tourist2d3 months ago

[dead]

hn-front (c) 2024 voximity
source