Low latency c++

[u/No-Introduction-2825]

I want to learn the ins and outs of low latency c++, so far I have read tour of c++, started reading concurrency in c++ (about 3 chaps done) and done a lot of competitive programming (is this irrelevant?). In your experience, is this the right way of approaching the subject? Is there a different better way? Any advise is much appreciated.

[u/RefrigeratorFirm7646]

stop reading, start coding. you cant learn to swim by reading swimming books. competitive programming is relevant but trust me most of low latency programming is about understanding computer architecture and squeezing out performance through cache optimizations and stuff, so that should be your focus

[u/No-Introduction-2825 → u/RefrigeratorFirm7646]

You are right. Although the main issue is, I come from a math background, so I never really studied computer architecture. Would you have any recommendations on how I go about the whole thing? Or how you did it?

[u/Kriemhilt → u/No-Introduction-2825]

You don't need to study "computer architecture", that's far too broad.

For low-latency on server systems you need to understand cache and pipelining, but these are likely to be irrelevant for embedded.

You also need to get comfortable with profiling (with perf, vtune, etc.) and benchmarking. The only way to make code faster is to measure it and iterate.

Good starting points might be "What Every Programmer Should Know About Memory" (either the original by Ulrich Drepper or Matt Godbolt's update if that's available anywhere), just searching for "low-latency C++", the LaurieWired video on DRAM refresh - any of these should generate forward links.

[u/mredding]

You need a mentor in your domain of interest. Learning on your own is a blind, random walk through the forest until you find your way. You waste a lot of time learning what may not be your focus or agenda, but you won't know until after the fact.

Barring that, look into Linux kernel bypass. It's the easiest to learn because it's documented and open source. Windows has similar, but I don't know anything about it.

You will also need to learn other concepts, like Data Oriented Design, Scatter/Gather, batch processing, DMA, memory mapping, CPU intrinsics, concurrency, serialization/marshalling, protocols, networking, zero-copy, single-pass, hot paths, fast paths, prefetching, branch prediction, and probably a few other things.

I don't know what you want to accomplish. Little's Law tells us about the relationship between throughput and latency, whereas Amdahl's Law tells us the limits of optimization (how much you can still reduce latency). Your biggest bottlenecks will be the network itself; if you can't control for that, then it doesn't really matter how fast your software is.

I was working for a prop shop for a bit where we had microwave tranceivers pointing out the window to the exchange across the street. The atmosphere is almost a vacuum, so light propagates through it faster than through fiber, and the fiber run down the building, across the street, and back up to the exchange is a longer path.

We also have FPGAs on the NIC cards so we can put the fast path on the chip right next to the ADC, so as the incoming signal is still coming in, we can already be generating the output signal of the response.

[u/No-Introduction-2825 → u/mredding]

Thank you very much. This seems like a very nice starting point. I will look into Linux kernels and the other topics you mentioned.

My aim is to expand on my knowledge after competitive programming, and targeting low latency c++ roles, if possible in HFT's one day, seemed like the right next step. So I wanted to build enough domain knowledge to do that.

[u/mredding → u/No-Introduction-2825]

HFTs are all in the FPGAs - anything that touches software up in the system IS the slow path; so learn finance and Verilog. This is more the domain of an EE than a software engineer. The competition in this niche is INSANE.

The idea of writing low latency software is romanticizing the trading industry circa 2010 and before - when we were all using conventional general purpose NIC hardware; because 2010 and after is when $20k-each purpose built commercial trading NICs became available. These things can't even support ping packets at the hardware level. Everything changed overnight.

And it's that romanticism that has me worried about you. I want you to dissolution yourself of this and know what you're actually asking to get yourself into, so that you might actually hit your target.

Additional food for thought: There are other industries and all manner of general purpose software that need or would benefit from low latency; maybe HFT isn't for you. Alternately, there are other industries that use these NIC cards, mostly these AI platforms that leverage entire data centers to deliver you a sub-second response to your prompt, so there's work for you beyond the HFT market, if you ever get bored or frustrated with it.

[u/Physix_R_Cool → u/mredding]

The competition in this niche is INSANE.

Do you mean competition for jobs, as in lots skilled people apply to these jobs?

Or do you mean that the people who do these jobs compete to write the best algorithms, so that their trades win out against their competitors'?

[u/mredding → u/Physix_R_Cool]

Yes to both.

Right now, the job market is tight, and that happened when Trump was inaugurated - shop operations went full conservative because with him, you don't know WHAT is going to happen from day to day, sometimes hour by hour. So job posts are listed, but these are companies filtering through the talent pool, looking for opportunities. No one is hiring for capacity; so you REALLY need to have something on offer. A lot of what I was coming across was companies were looking for patents and patent rights; so they're hiring patent holders to gain some sort of - ideally royalty free access.

And then the thing with currency markets is you're either the winner, or the Nth loser.

[u/Cultural_Act5304 → u/mredding]

Wow how wrong are you C++ is still valid in HFT, yes FPGAs are being used for hot paths but that does not mean c++ is not being used in hft. Most of the hft code is still in software and c++ devs are indeed needed in hft space. Also the competitive programing is a goot to have, but in hft space what you need most is knowledge of os, computer architecture and low level networking if you know these and you can program your code accroito this knowledge you are top 0.1%

[u/mredding → u/Cultural_Act5304]

I'm not incorrect, you're either misinterpreting what I said or misrepresenting it.

[u/esaule]

Really you need to understand the w hole path from signal to output. How does it go through the wire, through the processor, the cores, memory subsystem, through the OS. And you need to understand how your program will control for all of that.

[u/ste_3d_ven]

If you are willing to spend a little bit of money, computerenhance.com is a great place to begin learning about how to write performance aware code. This course goes through how the computer works at a low level with caches and pipelining, how to read basic x86 assembly, and even SIMD principles and programming. I can’t recommend it enough

[u/No-Introduction-2825 → u/ste_3d_ven]

Thank you. I checked it out. It does seem relevant, and the monthly subscriptions seem doable. Do they also have a good approach to learning that encourages you to write the code yourself rather than just following along?

Also, if possible, what would you say the prerequisites are?

[u/ste_3d_ven → u/No-Introduction-2825]

Yes! After most lectures he assigns "homework" and in the next lecture he goes over his answer to it. and I cant say there are much in the way of prerequisites. I would say you might want to have a basic understanding of C code to read most of his non-assembly code. But non of the code he writes is crazy complicated. Everything else he covers or points to external resources if you need it.

[u/bocsika]

One consideration: low latency business (high frequency trading) is much smaller niche than large throughput business (batch processing). Much more specialized, much stressful.

Worth considering that not only those areas are a good fit for c++, but e.g. medical image processing, embedded development etc.