I'm currently a CS student and considering switching to a Statistics major while keeping a CS minor.

My main concern is the current entry-level CS/SWE job market. I'm interested in AI/ML and possibly grad school eventually, but I also want to maximize my chances of having a solid career after undergrad.

I'm considering Stats because it seems like it could give me more options outside of traditional SWE, while still being relevant to AI/ML. But I honestly don't know if that's actually true or if I'm just rationalizing the switch.

For those with experience in CS/Stats or hiring:

Would you personally choose CS or Stats + CS minor if your goal was to maximize career options and minimize the risk of struggling to find a job after graduation?

I'd especially appreciate perspectives from people who have actually gone through the job market recently.

Welcome to r/MLSystemsDesign

This community is for practical discussions on designing and scaling production ML and AI systems.

Topics can include:

• ML training and inference platforms
• Search, ranking, and recommendation
• Feature stores and data pipelines
• LLM serving and GenAI systems
• Agentic AI platforms
• Evaluation, observability, and experimentation
• ML system design interview problems
• Real production tradeoffs and lessons learned

The goal is simple: go beyond model theory and discuss how ML systems actually work in production.

If you’re joining early, introduce yourself and share one ML system topic you’d like to go deeper on.

I'm currently training a small&fast model on trainign data that comes from a bigger&slower model. I treat the output of the bigger model as Pseudo Labels. But in the End, I need to evaluate the small model to Ground Truth labels. In that case, those are segmentation masks (Images with 10-40 segmentation masks). The pseudo labels are also far from perfect.
The question is also, how much they need to be improved (semi-automatic) to get acceptable results.

How many of the GT Labels do I need? Until now, I use 10k pseudo labels for training of the small model. It takes me 15-45min to do one GT labeled data by myself, so it's not feasible to have like 10% GT data for evaluation.

Or does someone know any keywords for that?

TLDR the question can be reformulated as: How many manually annotated samples are required to estimate segmentation performance with an acceptable confidence interval?

When people fine-tune their own local models, the model setup usually gets most of the attention. But in practice, a lot of the work seems to be on the data side.

If you are training on business data, you may need to bring in support tickets, internal docs, product specs, chat logs, code, policies, CRM notes, or domain QA pairs. And it usually does not work perfectly on the first run. Some samples are noisy, some are redundant, some domains overpower others, and some “bad-looking” examples are actually hard but useful.

One direction I have been thinking about is making the data strategy dynamic during training.

Dynamic selection means periodically choosing which samples should enter the next training window, using signals like loss, delta loss, gradient similarity, or external scores.

Dynamic mixing means adjusting the ratio between data sources during training, instead of fixing one static mixture before the run.

Dynamic weighting means keeping the sample in training, but changing how much its loss contributes to the gradient update. This is useful when you do not want to hard-drop uncertain samples.

This is the current direction in OpenDCAI/DataFlex: adding data selection, data mixing, and data weighting controls on top of the training loop.

For people here who fine-tune local models, how much time and compute do you usually spend on data preparation compared with the actual training run?

Where can i find this article?

I’m trying to figure out how to actually get a usable edge in the ML/DL space to get hired, but everything pushed to beginners right now feels like a trap.

For context on what I've done: I started off with Computer Vision, moved into GIS stuff, and recently went deep into the weeds of attention mechanisms and GPU kernel programming. I thought learning the hardcore, low-level math and systems stuff would set me apart.

But I’ve hit a wall. Let's be honest: no company is hiring a fresher to write custom CUDA kernels or design novel architectures. Those are senior research or PhD roles. The effort I put into the low-level stuff feels wasted because, for an entry-level dev, it's just personal trivia.

On the flip side, the standard "employable" advice is to build traditional ML projects (fraud detection, etc.) or slap together a LangChain PDF wrapper. But people have been doing this for years. Basic API wrappers are completely saturated and offer zero competitive edge. It feels like buying a stock after everyone already knows it’s going to go up.

So, what is the actual sweet spot between "PhD-level researcher" and "API wrapper"?

I want to avoid the YouTube influencer BS and focus on the real engineering trenches.

For the people actually hiring or working in the industry: what are the non-commoditized skills someone trying to break in should be grinding right now to have a real, usable edge?

(Note: The core thoughts and frustrations here are 100% mine, but I used AI to help structure and edit this post for clarity.)

Basically how to create a model that recognises such real world use case pattern, from data gathering (same or nearly similar data) to training the model to unfolding the output to get the text. Even synthetic data will suffice as long as the process is properly done, but synthesizing such real world level data is a project on its own.

Playing around with LLMs, Agents and GenAi for 7 years, I came to a conclusion: the new programming language is language itself in my case, English.

If you remove all the fluff (Stop words etc) and use none fluent English as a kind of Python-style syntax, something like:

“Read content from file then apply UPPER_CASE to all words”

…it starts to read almost like a functional call chain.

AI is pretty good at understanding programming language syntax.

What do you think? Is this question too stupid?

Edit:

I have 7 years of deep learning experience and llm/agnetic hands-on practice, so I mainly want to share what I’ve tried and learned along the way. That gives me a good understanding of both the inner workings and the practical side of using these technologies.

Hello, i'm in my hand a really big topic that i bet no one will be able to answer me.

i create a pipeline to generate prediction using clasp model.

The inputs are timeseries where the clasp models generate a vector of change points (predictions). After some process, i generate an output (vector of change points) and use an f1score to evaluate che prediction and generate a score.

My pipeline has different hyperparameters where different combination could change the outcome: so better to use gridsearch and cross validation to choose the best hyperparameters and then evaluate the model.

This is how i did:

Imagine you have only 2 configuration of hyperparameters: conf1, conf2

you divide the dataset in 3 different folds: a,b,c (dont point out about 20% or something, im just trying to make the example short as possible)

i generate prediction for ab, ac, bc with conf1.

i generate prediction for ab, ac, bc with conf2.

for both i evaluate with f1score the prediction and compute a mean. i found out conf1 is best.

i run again conf1 on all my dataset a,b,c compute the f1score, the mean and that's the score of my model. is this correct?

Im not so sure because this model doesnt have any fit or training. you just give into the input some timeseries and generate a prediction.

IF i had to use random forest, as we know, to evaluate properly a model, i would have to do cross validation. so

ab for training, c for validation = score_1

ac for training, b for validation = score_2

bc for training, a for validation = score_3

mean(score_1, score_2, score_3) = mean_score

easy right?

if you want to gridsearch, just execute an outer for loop to test each combination of hyperparameters and then choose the highest mean_Score for each combination of hyperparameters and thats it. easy right?

BUT HOW DID I DO THAT IF MY MODEL DOESNT HAVE A TRAINING?

if i repeat the process for random forest:

ab for training, c for validation = score_1

ac for training, b for validation = score_2

bc for training, a for validation = score_3

mean(score_1, score_2, score_3) = mean_score

so basically this means:
conf1, i generate prediction for a,b,c then compute mean

i do the same for conf2 and conf3 and then just select the highest mean? thats my model?

but then how do i test my model? the score you use to choose the best model isnt the score the model will perform on data never seen.

should i just randomly pick a fold and then use the rest 80% to find the best conf? but then what if im so unlucky the randomly pick test fold my model will score 0.0??? lmao???

so we need to do something like this https://www.kaggle.com/code/alexisbcook/cross-validation where you need to do cross validation to have a mean. so you are not unlucky and compute a mean.

so i compute a,b,c,d

a,b,c,e

a,b,d,e

a,c,d,e

b,c,d,e

and then compute for a,b,c,d,e when i find the best conf. BASICALLY AS I SAID I DID AT THE BEGINNING OF MY POST...

but is this correct?

Hi everyone,

I've recently finished the first version of Agentic GraphRAG Blueprint, a reference architecture for question answering over large document collections. I want to ask what should I improve in my project?

Instead of plain chunk retrieval, it builds a knowledge graph combined with vector search, so answers can connect facts across documents.

Key features:

• Incremental ingestion - unchanged files are skipped via content hashing, and community reports regenerate only for affected communities, keeping token costs low as the corpus grows.

• Hybrid search - local mode for fact-level answers, global mode for cross-document synthesis.

• Domain-agnostic LLM prompts - easily swapped via PROMPTS_PATH, with Leiden-based community detection.

• Deployment - run it locally with Docker or provision everything in the cloud with Terraform and CI/CD.

Link: https://github.com/sebastianbrzustowicz/Agentic-GraphRAG-Blueprint

I'm looking for any feedback. What can I improve?

Hello everyone. ML beginner here. I have the basic understanding of ML and have been given a project to create a model through which we can predict cpu metrics so that we can proactively monitor cpu spikes before it creates an incident. I’ve been using Claude to help me out here and it suggests to use XGBoost for this. But the accuracy is not up to the mark. Can anyone help me out here if you have worked on similar projects. Thanks for the help in advance

Hi everyone,

I’m a student working on a research project on risk-aware GitHub PR review. I’m doing the project mostly on my own and I don’t have access to a research lab, large compute budget, or people who can manually annotate thousands of PRs, so I’m trying to find a practical approach that I can actually finish.

The idea is to take a GitHub PR and estimate four types of risk:

1. Bug / correctness
2. Security
3. Compatibility
4. Cross-system / integration

The architecture I’m working with has four separate risk models. They share the same PR characteristics/features, but each risk model has its own historical data, prior, and evidence.

My main problem is the historical data needed for those priors.

At first, I looked for a single PR dataset where I could get reliable PR-level outcomes for all four risks. I couldn’t find one.

I then tried looking for separate datasets for each individual risk model. I thought this would solve the problem, but I keep finding datasets where the labels look relevant at first but don't actually represent the outcome I need.

For example, SEVRA-plus looked very promising for the Security model:

https://huggingface.co/datasets/RedAI4Code/SEVRA-plus

It contains security-related PR examples with vulnerability/CWE information, but the malicious PRs are deliberately constructed by reversing real CVE security fixes. So although they are useful for evaluating or studying security vulnerabilities, I don't think I can use their class distribution directly as a real-world prior for ordinary GitHub PRs.

I’ve run into similar issues with other datasets:

- some label the linked issue rather than the PR implementation,

- some label review comments rather than actual PR outcomes,

- some contain artificially constructed vulnerable/failing PRs,

- some only give merge/close status, which doesn’t tell me whether the PR itself was buggy, vulnerable, incompatible, etc.

The distinction between the issue and the PR is especially important for what I am trying to do.

For example, imagine a maintainer opens a security issue, someone creates a PR to fix it, but the PR implementation itself contains a correctness bug and gets rejected. For my problem, I need to know the nature of the PR, not simply inherit the security label from the original issue.

Similarly, a PR could be opened to fix a small bug, get merged, and then later cause a compatibility problem. Again, I care about what happened because of the PR implementation, not just why the PR was originally opened.

Because I couldn't find a dataset that directly gives me what I need, I tried a practical compromise.

I took 96 real PRs from SWE-Review-Chat, filtered them for sufficient evidence, and used an LLM to annotate the four risk states from the information available in the PR record, such as the description, review discussion, diff context, tests, and lifecycle information.

I’m treating these as weak/model-assisted labels rather than independent ground truth.

The resulting usable outcomes are:

Bug:

30 present / 9 absent

Security:

1 present / 7 absent

Compatibility:

5 present / 12 absent

Cross-system:

4 present / 8 absent

So now I feel like I’ve hit a wall.

I can keep searching for datasets, but so far I haven't found anything that solves the underlying problem. I also don't have the resources to manually establish reliable ground truth for thousands of PRs.

I’m therefore looking for advice on "how I should move forward from here".

Should I continue with the small real dataset I have and explicitly model the uncertainty caused by the sparse risks?

Should I rely on LLM-assisted annotations of real PRs as a practical research compromise, or is there a better low-resource approach that I am missing?

Or is there a completely different way of constructing the historical priors that would make more sense for this problem?

I’m not looking for a perfect dataset at this point. I’m mainly looking for a practical and defensible way to move forward given that I’m a student doing this alone with limited time and resources.

If anyone has worked on GitHub PR datasets, Mining Software Repositories, empirical software engineering, code-review research, or Bayesian risk modelling, I would really appreciate any advice on what you would do in this situation.

Thanks!

Hey everyone,

I have an upcoming Data Scientist interview with an ML system design round. I asked the recruiter what to expect and they said it’ll be high-level ML system design, not traditional SWE/low-level system design.

They mentioned focusing on things like:

• Problem framing
• Data/model considerations
• Evaluation
• Productionization/deployment
• Monitoring
• Tradeoffs when designing ML systems

Basically, it sounds like I’ll be given a real-world ML problem and have to explain how I’d approach it end-to-end.

This is my first dedicated ML system design interview, so I’m not really sure how deep I should prepare.

For people who’ve done these interviews:

How did you prepare? What resources did you use? What kind of questions were you asked?

Also, how deep do they usually expect you to go into things like feature stores, model serving, APIs, streaming, retraining, etc.?

Any good resources, YouTube playlists, GitHub repos, books, or example questions would be really appreciated.

Thanks!

I'm looking for a potential AI solution for something I'm trying to accomplish with my business. The large majority of customer communications are through employee emails and I'm hoping a solution exists that may be able to track specific situations and send notices to me if those scenarios occur. For example, customers checking up on delivery dates or adding items to orders last minute. To take it even a step fruther, if it could notice when an employee is responding rudely to a customer (or vice versa). Our number of employees and volume of emails is too high to track through our normal system. I am not an AI expert so if anyone has suggestions or can point me in the right direction, I would appreciate it.

I'm a Data Analyst, and I'm trying to bridge a gap in my Data Science understanding.

I know the concepts behind classical ML reasonably well but I want to understand what actually happens to an ML project in a real production environment from start to finish. I want someone to walk me through a real project in terms of:

We use this application/tool to do this → it produces this output/file/artifact → that goes into this tool or system → then this team works on it → then it moves to the next stage.

For example, where do we actually write the code—Jupyter, VS Code, Databricks, or something else? Where does the data come from, and which tools are used to extract and process it? Once the model is built, where is it saved? How is the code tested? How does Git fit into the workflow? Where do MLflow, Docker, FastAPI, Airflow, CI/CD, Kubernetes, and AWS/Azure come in?

Basically, I want to understand the actual sequence of tools used in a real production ML project. If you work in Data Science, ML Engineering, Data Engineering, or have worked on real client projects, I would really appreciate it if you could explain the actual end-to-end stack used in your organization through one practical classical ML example.

Would really appreciate detailed answers from people with real production experience.