I wanted to see if I could use Python to predict commodity markets. I gathered daily price and volume data for Orange Juice futures since 2002 and pulled in weather data via Open-Meteo. I used XGBoost to create a classification model and ran a grid search to test different parameters. To evaluate it, I used a confusion matrix and AUC scores, eventually tuning a parameter to only trade when the model was highly confident. It’s a fun, slightly ridiculous project that ends with me actually buying a 10:1 leveraged futures contract. Check out the build process here: https://youtu.be/5q9-8F9sxkw

I want to add weather forecast data because I expect weather is priced in by the time it happens. Does anyone know of a free api for this?

I am working on my strategy but i couldn't find a proper way to catch pullbacks. Most of the time the trade goes against me at the last step and i am trying to find a great way to determine pullbacks. I tried various of thing but none of them were the thing i needed. I am open to any suggestion

My last post got a lot more attention than I expected, and quite a few people asked what I was going to do next. A few also asked if I could share at least some of the strategy.

The original backtest ended on July 31. I'm leaving the strategy exactly as it was at that point and using the full month of August as a forward test.

I haven't looked at the August data or used any of it to adjust the strategy. No parameter changes, no new filters, no fixing things after seeing the result.

I'm not just adding August to the original Jan-Jul backtest either. I'm running it separately through the forward-testing side, and that's giving me a technical problem at the moment. Processing the full universe at 1-second resolution is running me out of memory. I'm working on that now, so I don't have the August result yet.

The run will include the corrected broker costs, fees and slippage assumptions as well.

I still don't want to post the exact rules or parameters, but I can explain a little more about how the strategy works.

RVOL comes first, but RVOL itself isn't the entry signal. I use it to judge the current state of the stock and narrow down what the strategy should be paying attention to.

From there I'm watching the 1-second data for a specific value. When that value breaks a certain level, that's what triggers the buy. It's something I'm specifically measuring at 1-second resolution, which is also why this strategy isn't based on minute bars.

The exit is a little different.

Once I'm in a position, I divide the stock's behavior into several states. Those states are checked continuously after the entry. If the stock moves out of the state where I want to keep holding it, the strategy exits immediately.

So RVOL gets me to the stocks I'm interested in, the 1-second data determines the actual entry, and what happens after the entry determines when I get out.

That's about as much of the strategy as I want to give away for now.

Once I get the memory problem sorted out, I'll run August without touching the strategy and post the numbers. Good or bad.

If you have over 200K trades in a year (I'm projected to have ~1M trades this year) how does one report it to the IRS? That would be several tens of thousands of pages of attachments on 8949.

I do not have the MTM election and do not wish to since I also have long-term investments and want the 15% rate for those

Has anyone been able to make a profitable intra day signal from XGBoost. Most here and “the machine” recommend using it to supplement the signal from another strategy. I’ve been trying to use 10 minute windows of one minute ohlc bar data with very poor results. I do use dimensionless or normalized features.

I just created a strategy which beats buy and hold for some top 10 crypto assets.

Btc by 114% (aug 17 2017 - today)

Eth by 32240% (aug 17 2017 - today)

Bnb by 23% (nov 6 2017 - today)

Xrp by -25% (may 4 2018 - today)

Sol by 71% (aug 11 2020 - today)

Trx by -64% (jun 2018 - today)

Zec by 200% (mar 21 2019 - today)

Unfortunately i cannot adjust timeframes as it requires Ultimate plan

I tested RVOL at the open and found that higher RVOL was pretty good at identifying stocks that were likely to make a large move, but it didn't tell me much about direction.

I've done quite a bit more testing since then, and this is one of the results.

It's a systematic US equity strategy using 1-second data from Jan 2 through Jul 31, 2026, trading between 9:30 and 11:00 ET.

I'm keeping the exact entry rules and parameters private, but RVOL alone isn't the entry signal. There are other conditions involved.

Results:

7,137 trades 63.6% win rate PF 1.84 +0.17% average trade 6m 7s average hold -0.9% max drawdown

I ran it locally on stored market data using my own backtester. For fills I'm using quote/trade data, not just assuming a fill because a bar touched the price.

The 0.9% drawdown is the number that bothers me. It seems too low considering the number of trades and the return.

I've been checking the fill logic, costs, lookahead, position accounting, overlapping positions and universe selection to see if something is making the result look better than it really is.

Has anyone here had a backtest look this clean and later found something they had modeled wrong?

Any good app to predict market crash?

Hi all, I've been working on Algo trading for about 6 months. Currently paper only and planning to slowly migrate some of my bots to real money in a month or so. I have a question for you guys - i am working with an assumption that most edges are finite, after 4-5 years high chance anything will be arbitraged. With that being said, ​how do you deal with backtesting? I have been testing in era 22-present and backtesting OOS 2011-2021.

If strategy works in both eras, it's promoted to next round of analysis. However, if it works on current era but flat in backtest, for me this isn't necessarily a kill since different eras have different edges. ​I still do further analysis but I prioritize current wins era vs OOS wins.

Do you guys think this is wrong? How do you handle this conundrum?

Have been trying to refine the bot since I moved from paper to live in February. Getting a decent Beta since the last 3 weeks so started publishing my daily list on substack.

I think the watchlist is more important than the bot's entry and exit time.

Looking for advice on what next strategy should I use - currently only using Momentum.
Is it a good idea to start buying options using algo ?

Just heard someone say that most trading bots/algos trade the reversal, not the continuation. That sounded suspicious to me. Is there any truth to that?

Spent part of this summer reconciling a signal pipeline against what actually happens at the broker, and the gap between "backtest fill" and "my fill" breaks down into more categories than slippage. Sharing the categories because I'd like to hear which ones bite other people.

The obvious one is delay. A monthly system computes at the close, a human trades the next session or later. We measured that on 3 published tactical strategies: 1 session late is mostly survivable, 2 sessions late costs about 1 CAGR point on the faster 2. Slippage gets all the attention in backtest threads, but for monthly systems the calendar slip dominates the bid-ask by an order of magnitude.

The nastier category is data basis. Same ETF, same day, a momentum score of +0.89% on total-return data and -9.61% on price-only data, because the fund yields enough that 12 months of distributions flip the sign. Neither series is wrong. A backtest on one and execution logic on the other silently trade different strategies, and nothing errors.

The one I still don't have a clean answer for is intra-bar signals. On the same day as that split, the deciding price crossed the signal threshold twice on 30-minute closes and finished 22 cents above it. A monthly rule only cares about the settled close, so discipline solves it. But every "check the signal at 3:50 PM and pre-position" scheme I've tested is a different strategy than the one backtested, and usually a worse one.

For those running live systems: what's in your reconciliation checklist beyond fills vs assumed prices? And has anyone built automated detection for the data-basis class of bug, or do you find it the way I did, by staring at 2 numbers that should match and don't?

I stream lvl2 data via interactive brokers and I'm looking for cheap alternatives as I'm facing many issues including a 3 concurrent symbols limit at any given time, which limits how many symbols I can be viewing simultaneously.

Besides, their shitty api in addition to being cumbersome and tedious to use, does not even work properly and occasionally prevents unsubscribing from symbols, which means I cannot just close some tabs to open new ones with new symbols, I have to manually restart the crappy IB Gateway for things to work again.

I'm currently evaluating alternatives. Webull, Moomoo, TOS, Tradestation, and probably most of the available options don't work at my location. The others available are pretty expensive ex: databento ($200/month).

I was checking Nasdaq TotalView and it's offered for $15 (non-pro), and I was wondering, what's the difference between this subscription and obtaining data via a broker, and what explains the difference in price? I mean why do I have to pay $200+ in some places while I can get the same data for free on TOS, $2 on Webull, $15 TotalView (if it's the same data).

I was looking at execution costs in low-priced US equities and found something I didn't expect.

Stocks below $1 can quote in $0.0001 increments, while $1–$5 stocks generally quote in $0.01 increments — a 100x difference in minimum tick size.

But the finer grid didn't produce tighter markets.

I sampled 38 regular trading sessions between April 2025 and July 2026 using one-second BBO recordings:

276,345 quoted seconds below $1 2,047,344 quoted seconds from $1–$5

Median spread:

Under $1: 1.81% $1–$5: 0.72%

So the group with the 100x finer grid actually had a 2.5x wider median spread.

The more interesting result was whether the minimum tick was actually binding:

$1–$5 stocks sat at their one-cent minimum 60.4% of the time Sub-$1 stocks sat at their $0.0001 minimum only 2.4% of the time The median sub-$1 spread was 96 minimum ticks wide

My interpretation is that once the tick becomes small enough, it stops constraining the spread. At $0.50, one $0.0001 tick is only 2 bp, so the market can quote dozens or hundreds of ticks wide.

That also raises an execution question: improving a $0.50 bid by one tick costs almost nothing, so stepping ahead of resting limit orders is extremely cheap.

I haven't tested actual fill quality yet — this is quote data, not fill data.

For anyone modeling passive execution in sub-$1 stocks: how do you handle queue position? Explicit queue modeling, empirical fill probabilities, or something else?

I run a systematic volatility strategy. not HFT, just a daily rebalance plus occasional intraday adjustments, but it does need to catch moves in pre/post market, so IB Gateway has to stay up and connected basically around the clock.

I've been looking at VPS providers that specifically market themselves for IBKR/algo use, and AlgoVPS's product page is by far the most detailed I've found on the actual pain points - they claim to handle TWS/Gateway's forced daily restart automatically, and are upfront that you still need to approve 2FA on IBKR Mobile about once a week rather than daily. Sounds like what I need, except I can't find a single independent review of them anywhere. Which is making me nervous given I'd be trusting them with something that would run unattended for weeks at a time.

So, questions for anyone who's actually used them (or considered them and went elsewhere):

• Has anyone actually run IB Gateway on AlgoVPS? Does the auto-restart handling work as described, or are you still babysitting it?
• What's real uptime/latency been like, not the marketing number?
• What's support actually like?
• Anyone hit surprise billing issues, or had a clean experience getting a refund/cancelling?
• If you passed on AlgoVPS for something else in this space (QuantVPS, TradingFXVPS, TradoxVPS, NYCServers, etc.), what tipped it for you?

I’m planning an experiment with an AI agent controlling a small, isolated brokerage account. This is not retirement/core money, and I’m completely prepared for it to underperform or blow itself up.

The basic objective:

“Maximize the terminal value of this account over 5 years.”

I deliberately don’t want to specify an investment strategy. Choosing and adapting the strategy is part of the experiment.

Current hard constraints:
- No margin or borrowing.
- Account/cash balance cannot go negative.
- No withdrawals or transfers out.
- Contributions follow a predetermined schedule; the agent cannot request more capital.
- Every trade and material decision must be logged with an explanation.
- Performance is benchmarked against passive buy-and-hold receiving identical contributions on identical dates.

I want to distinguish between operational/safety constraints and investment-strategy constraints. My goal is to specify the former while leaving the latter as open as reasonably possible.

So:

1. What safety/operational guardrails are essential?
   
2. What constraints would you NOT impose because they start dictating strategy?
   
3. What loopholes could produce hilariously stupid but technically compliant behavior?
   
4. How would you benchmark this fairly?

If five years from now the conclusion is “the AI did a tremendous amount of work to lose to buy-and-hold,” that’s a perfectly acceptable result.

Basically: give it a destination, fence off operational stupidity, and otherwise let it cook.

Afaik no Indian broker (zerodha, angelone, upstox, fyers) provide bid ask values for historical data.

For most of 2026 I ran an automated ICT-style futures strategy (1h sweep -> 5m MSS -> FVG entry) on two prop eval accounts from a VPS. The backtest behind it: +1,663R on NQ over six years, profit factor 3.03, with a 2.8% max drawdown at 0.5% risk per trade. It had survived a train/holdout split, walk-forward across 13 half-year windows, a look-ahead audit, fourteen parameter studies with control arms, and a somehwhat robust test suite that ended at 815 tests.

Broker statement after seven weeks live: 32 positions, net ≈ 0R (+0.03% of account). The config actually deployed at the end: 9 positions, 1 winner, −2.6R.

The cause was one line. The simulator started checking the stop on the bar AFTER the fill (commented "avoids instant stop-outs"). The stop was placed under the pivot of the bar the entry fired on — a bar whose full range the simulator already knew. A stop anchored under a wick, exempt from the bar that made the wick, cannot be hit by the move that defines it. Let the fill bar test the stop and 64% of trades die on their
entry bar; +1,663R becomes +76R, and stripping the top 8 winners leaves −312R.

The part that still bothers me: fourteen studies didn't catch it, and they couldn't have. Every study compares an arm against a control on the same simulator — an artefact shared by both arms cancels out of every comparison. Walk-forward passed it because the artefact was uniform in time; walk-forward validates stability, and arithmetic is very stable.

The broker settled it. Half my NQ entries were stopped within five minutes of filling, three of them in 0–5 seconds. Matching the honest simulator's per-trade verdicts to the broker's fills: model-says-loss → broker-loss, 20 out of 20.

Full write-up, every figure reproducible from committed files:
https://honestbook.fly.dev/writeup

It ends with the 8 things I'd measure before buying another eval, and links a free calculator I put together that computes expected payout per fee dollar (a ratio, not a projection) under a firm's actual rules at your stats. No signup, no affiliate links — I don't make money from any of this. I mostly wrote it so the next version of me finds it before spending the two years ✌️

Hi guys just looking for a dev to help with something really simple with an ea which is connecting the .mqproj to the actual ea, as in connecting the codes together so the two files can read.

Its just a quick connection so I am willing to pay 2-3$, no crypto but let me know what form of payment you'd like.

I will provide the mql5 source code itself and also the .mqproj file. You can also just record a little video showing me how to do it.

Designed two bots one long only and the other short only bot, this bot which works on all exchanges but for some reason it does better on Bybit followed by OKX! It’s a Grid Long Only Bot, I still think that I can squeeze some more juice out of it though 🥵 (Hello Greed).

For decisions on when to turn on either of the two bots I use a structure monitor! Trying to build a bot that performs well on bull and bear markets it’s a daunting task!

Hai guys,

I am about to go down into a rabbit hole. I just start learning to prompt here and there to find profitable strategy utilising liquidity concept. I am a big fan of ICT for its silver bullet model, SMT,SSMT ,etc.

I would like to see advice from those that have done the samething whether this is good hole for to deep or not.

Thnx in advance

been hearing about it and curious about opinions.

I only started this for fun, as an experiment to disprove trading but I got hooked when my strategies started making thousands on paper over a few weeks. I have now put 6 months into developing a bot and I am in over my head, as AI has allowed me to develop so much so quickly, and my apparent edge quickly disappeared.

I build my own candles and have 6 months of ticks. I am spreadbetting¹ (I'm in the UK). I have my own back-tester, which has 6 strategies² I started with Gold and Brent Crude. Other assets including shares are possible but I will not be able to download candles for back-testing in any significant quantity.

I really got into this when it was making £000s as the Middle East was blowing up, so my next phase was to implement a simple regime filter. It collected 60 days of paper trades and recorded average R, win rate etc for each strategy/epic and the plan was to deploy each strategy in the cells where it was profitable³. I didn't see the profits that I had at the start, either because the markets stopped reacting as strongly or because my edge was just an illusion ^a.

Next, I tested different ways of classifying the market regimes according to different variables that were relevant to each strategy⁴. This did not find any area where a strategy was much more effective that when deployed in general.

Having also back-tested lots of parameter changes and new strategies, I concluded from my failure to find a market-state in which a strategy was profitable, that I have failed - or to put it another way - succeeded in proving that day-trading doesn't work.

I have a problem with this conclusion: I don't like it. 1. It appears to go against a number of Redditors who say they are profitable. 2. I enjoy trading and would like to do this full time. 3. I don't feel it's a sound conclusion.

So my question is: Where do I go from here?

I could trade a different asset (shares/ETFs), pair-trade assets, move to swing trading from intraday.... There are so many options.

Footnotes:

1 - One thing to consider is whether my tick stream is unreliable ^b.
2 - Candlestick patterns with a higher-timeframe trend gate, Bollinger Band mean-reversion, BB Breakout, moving average crossover, trend pullback and some others. The strategies also control the stop/limit, trailing stops if used, Kelly bet sizing...)
3 - Cells= Regime (Trend, Mixed Trend, Chop), Direction (Buy/Sell), Turn? (whether shorter timeframe agrees with trend)
4 - Various combinations of Hilbert Transform/Trend, Velocity, Spread, Bandwidth expansion rate, ADX ...

Notes to self:
a - check how the regime classifier would have classified those early days and see if perhaps we could have an 'extreme' trend to keep those days separate - if the maths supports it. Also, I wanted to check whether the additional timeframes coming online were triggering more contradictions.
b - [deleted to avoid triggering automod].