Spent the first half of my 20s doing odd jobs and finishing grad school. Since then I’ve been fortunate to work low stress jobs with good work life balance and moderately high pay.
Now that I’m in my early 30s, living in a high cost of living area with a strong job market, and socializing more, I’ve realized I don’t come close to making what FAANG people make. I can’t help but wonder if that’s something I should be chasing or at least exploring. At the same time, I really value how low stress and stable my job is, even though it pays well below market rate.

For those who chose not to go into big tech/FAANG, do you have any regrets?

Recently received an offer to go work at McK/BCG/Bain as a forward deployed AI Scientist. Interested to hear thoughts on what people think about DS at these consulting companies.
Do they do fun work?
How cutting edge are they?
WLB?

Currently at a large UK Financial services firm. Is this considered an upgrade?

TL;DR: My DS/analytics role has expanded into senior-level data/platform engineering and client leadership, but my title, pay, and promotion path haven’t kept up.

I’ve been in data science/analytics for about 11 years. Most of my earlier career was at a Fortune 100 financial company, where I eventually became a Data Science Manager and led a small team forecasting risk metrics that fed into public earnings reporting. I’m now a Big Data Analytics Manager at a fintech/fraud prevention company, working remotely in the US.

The reason I’m posting is that my job has changed pretty dramatically from what I was hired to do, and I’m having trouble figuring out what the role actually is anymore. My official job description is basically an Implementation Manager description with some analytics language added. It says things like “leverages tools built by the Implementation Manager to analyze big data” and asks for proficiency in Python, Spark, or SQL.

That’s pretty far from what I’m actually doing now. We process 1B+ transactions a year, and I’m working on a novel, high-visibility real-time fraud use case for one of our three largest clients. They’re also notoriously difficult to work with.

On the technical side, I’m adding custom platform capabilities for data ingestion and operationalizing ML models, building secure pipelines, doing Spark/PySpark processing, shell automation, SFTP workflows, and building reporting systems that run essentially autonomously. I’m hands-on with almost all of that work, but I’m also project managing the data engineering effort across both companies, coordinating our teams with the client’s technical teams to actually get this stuff into production. Then I’m still doing the analytics on top of the systems I built.

None of my previous responsibilities really went away either. I still manage other technical projects, work directly with the client, and regularly present analytical insights and financial reporting to their executive leadership. I was also heavily involved in work that helped roughly double the size of this client’s contract, which in turn expanded my scope further as we added products and took on more of their transaction volume.

So I’ve ended up doing some weird combination of data science, data/platform engineering, analytics, reporting, project management and client leadership. I actually like the engineering work, so this isn’t a complaint about having to code. I’m more confused about how a role that was originally defined as basically implementation + analytics ended up owning this much production engineering, platform work and client delivery without the classification changing.

The leveling side is where it gets stranger. My boss specifically encouraged me to interview for a Senior Manager opening on our own team. He’d been giving me very positive feedback, telling me he trusted me and that I’d have opportunities, so I went through the full interview process and eventually made it to the VP. During that interview, the VP said something along the lines of, “Well, who else would we hire? You’re already doing the work.” Then later in the conversation he asked whether they’d have to backfill my current position if they promoted me. I ultimately didn’t get the job.

I obviously have no way of knowing whether that question was decisive, but the timing has always bothered me. They left the Senior Manager role open for months and eventually filled the need with another person at the same level I’m currently at rather than hiring a Senior Manager. When I later raised compensation with my boss, the feedback was still positive, but he said I was progressing within the “normal range for my role.”

For additional context, I’m at about $129k base / $148k total comp and I’m already well below the midpoint of the salary band for my existing role. That’s part of what made me start looking harder at whether the role itself is even classified correctly.

For people who have been around senior DS/data organizations, would you still consider this a data science/analytics management role, or at this point is it really some form of data/platform engineering or technical data leadership? I’m also curious how you’d interpret the promotion sequence. Am I reading too much into the backfill question, or does this sound like the classic problem of becoming more valuable in your current seat than the company wants you to be somewhere else?

I'm a Data science manager at a mid-sized company. Every week I have a bunch of meetings, and my reports do all the data science work. They even manage their sprints.

My question is, what do I do? I have free time in my calendar I dont know what to do with. It feels like I'm a viewer in a movie

Hello all, here is a chill post.

When I was an undergrad, I really liked working things out on a whiteboard. Drawing stuff, talking through ideas out loud, testing little hypotheses.

Now I work in radar DSP, and a lot of my work is code, numerical experiments, deep learning and waiting for training to finish 😅

I’m wondering how other people bring that whiteboard style of thinking into DSP, data science or ML work.

Do you still use a whiteboard regularly, or do you mostly go straight from idea to code?

I recently left my first data science job that I had for a year out of my undergrad. The team wasn’t that great and faced a lot of issues with very high turnover relating to poor culture and rudeness, and just very low emotional intelligence.

I had one coworker who would always make really rude comments, he had just finished his masters and it was his first full-time job. he would say things like even though we’re all the same title and just started I’m senior to you and you and so and so because I make more money. there was another coworker who I had like an interview coding competition for the job and I won it so I got the job, but then later on they decided they needed more help so they hired him after and this coworker would refer to him as like the intern or the second choice. he would tell certain people on our team that they shouldn’t be working the job and that they weren’t ready yet. all this was very unprovoked no one ever said anything to him to make him say this stuff. In my opinion he was kind of a spoiled brat from like rich parents who paid like a ton of money to do a masters here and then in my opinion I think that he was maybe a little disappointed with his outcome, but yet no one ever said anything to him or prompted him to make rude comments. when we hired certain managers, he would say things to me like I didn’t think we should hire them they didn’t seem so impressive after we did I would just be like dude what’s wrong with you? I never knew how to handle this. I never thought about tattle tailing on him or anything, but it was just so annoying to have to deal with it and feel so helpless looking back. What would you do in this situation besides just like leaving the job maybe?

I've started my career in nonprofits and only worked in nonprofits until now. 3 times now, I have ended up in roles where I am the ONLY analyst on the team. Everyone I work with is either data adjacent, or not an analyst at all. I'm the only person ever working on analytics work, and I have no real life gauge/context on how to do things better in a real world context. I google things all the time, I take courses, but the advice is too generalized and doesn't go deep enough. I need people I can bounce off of. My biggest hope starting as an early career data analyst was that I'd be able to learn from other analyst and fill the gaps in my education with knowledge from mentors.

Instead, I have people looking to me to be an expert in analytics just because I'm the only one available(as if I'm not a junior). Very few opportunities to learn from actual analysts and get experience from them instead of the generalized advice online. I feel like I'm being stunted, but its incredibly hard for me to find roles that are placed in analytics teams, or where I'll be working under a senior analyst (and not just a VP or project manager). Have I screwed myself? Why is it seemingly harder to find analytic roles that work with other analysts?

Welcome to this week's entering & transitioning thread! This thread is for any questions about getting started, studying, or transitioning into the data science field. Topics include:

• Learning resources (e.g. books, tutorials, videos)
• Traditional education (e.g. schools, degrees, electives)
• Alternative education (e.g. online courses, bootcamps)
• Job search questions (e.g. resumes, applying, career prospects)
• Elementary questions (e.g. where to start, what next)

While you wait for answers from the community, check out the FAQ and Resources pages on our wiki. You can also search for answers in past weekly threads.

Article

I'm a table top game designer that used AI to build playtesting models. I previously wrote How to Train Your AI Dragon and The Artificial "Intelligence" of Artificial Intelligence which some people here might have read

I really enjoyed writing those articles so decided to enter the writing competition at King's College London to write even more about Machine Learning in game design

My usual style is to mix humor with technical content. I actually wanted to look at the more philosophical side of AI. Looking at why model outputs model, what sort of outcomes AI cannot model, and whether AI actually accomplishes anything important

New oss relational learning benchmark, leaderboard and TabPFN harness

1. RelArena-α: standardized relational machine learning model benchmarking
2. TabPFN-Rel: a harness for tabular foundation model TabPFN-3 for predictions over relational data
3. RPI-α (Relational Prediction Interface): an interface to run any RelArena model on your own database

- RelArena is open sourced here: https://github.com/PriorLabs/relarena

- Full report: https://arxiv.org/abs/2608.16319

- There's also a higher-level summary of the release by Prior Labs: https://priorlabs.ai/blog-posts/introducing-relarena?utm_source=socials&utm_campaign=relational

I feel like I am googling everything but the exact term I need, and would appreciate someone pointing me in the right direction.

Say you have a customer churn model. You predict a customer has a high likelihood of churning, and then the customer service team gets an alert to intervene. Great! Your model helped mitigate a loss and contributed real value. This is where most tutorials or blog posts on models like this end.

But overtime, customers that have all the signals of churning begin to out perform their expected value.... which would screw up your training data. You've succeeded in putting your thumb on the scale, but in the process potentially damaged the viability of your model.

What is the technical term for this phenomenon? Feedback? It's not target leakage I don't think. Googling "customer churn feedback" just gets you articles about using customer feedback forms as a predictor of churn, which isn't what I want.

Thanks!

I know data scientists are increasingly being asked to own models end-to-end. The problem I see (specially with juniors) is that they jump straight into Docker or other MLOps tools without building the foundation first.

I’ve been in data science for 8+ years and I think the best way to start is with ML system design.

I’ve gone through different resources over the years like Chip Huyen's "Designing Machine Learning Systems" and realized that learning system design just by reading a book or staring at diagrams is really hard. You don't really get the intuition until you can see how the components actually connect and behave.

So I built a free interactive tool based on a real system I deployed. It walks you through how the system was architected so you can build the intuition to design one yourself.

Here it is: https://futureproofds.com/tools/ml-system-map

A few things you can do with it:

1. Play a flow and watch it run step by step (training, serving a live prediction, the nightly batch, a drift alert firing)

2. Click any component to see how it works and why it matters in the system

3. Follow the build order to see how the whole system comes together at different stages

Hoping some of you find it useful. Would love to hear what you think, and let me know if there's any functionality you want me to add.

I was given a home assignment to do modeling for some adtech data. They had no explicit ask about what kind of model or how deep you have to go. Just data and they asked we want to see the modeling.

I spent lot of time in understanding the data, identifying the features, creating labels etc. When it came to modeling I picked Catboost since they handle categorical features quite well. I even mentioned how this can be further tuned and/or different models can be compared. I put it explicitly in a section for future work. Finally this was the thing that got me rejected.

Basically they expected me to compare different model families from more complex deep models to such boosting models. I have worked in this domain and actually such models (catboost) works quite well. You don't need very complex models. I remember in one of the previous jobs, they had like ensemble of 3 deep models which was super slow and was so painful to maintain. I basically replaced that with a boosting model + some probability calibration which did quite well. Also the data size I got for the task isn't big enough to justify such huge models.

In any case, I wish these tasks would be more explicit in what they are looking for. I know they also want to see how I handle ambiguity but it's really hard to assess which side of it is worth handling since I am not building a full fledged system. I explained all the decisions I made and why I did it. Also what I didn't do and why.

I see posts like these on my Linkedin feed every day. At this juncture, I am not sure if this is true or just one of those AI Slops - I am assuming there's a grain of truth in them.

But now, when I am preparing for interviews and job hunting, I don't think I could have ever imagined answering it in this way, unless I have worked on specific/adjacent use cases.

How does one prepare for such questions?

Welcome to this week's entering & transitioning thread! This thread is for any questions about getting started, studying, or transitioning into the data science field. Topics include:

• Learning resources (e.g. books, tutorials, videos)
• Traditional education (e.g. schools, degrees, electives)
• Alternative education (e.g. online courses, bootcamps)
• Job search questions (e.g. resumes, applying, career prospects)
• Elementary questions (e.g. where to start, what next)

While you wait for answers from the community, check out the FAQ and Resources pages on our wiki. You can also search for answers in past weekly threads.

I’m a Data Science student (career changer, not in a data related role). My program is focused more on the applied statistics side, so most of my classes use R. I’m already familiar with Python since it was the main language used in my prerequisite courses, and I’ve completed projects using Python, so I’m comfortable with the syntax.

However, I’m really enjoying using and learning R in my classes and seeing what it can do. Many of the statistics textbooks I’m interested in use R as well. I’m starting to explore R more deeply on my own and plan to start using it for personal projects.

But I’m curious, is R still used in industry? I know it’s heavily used in academia. I also know that in the current AI/ML world, Python is used heavily, which is the main reason I use it for all of my personal projects at the moment.

I’d like to eventually be comfortable with both and take advantage of the strengths of each language. But, of course, there are also people who say learning R is a waste of time.

I have a 30minute zoom meeting for a data science job and I'm just wondering what types of questions others have been asked in these interviews?

I had one a couple months ago and they did ask me a SQL question but that was the only technical one I can remember

Edit: Interview finished and doesn't look like I got it y'all! I'm a fucking idiot! There was absolutely no technical questions, just "Tell me about yourself" "What's your experience with python" "Walk me through a project" "Do you use Generative AI"

I'm not entirely sure how I messed that up but I guess my charisma stats are that low

PhD computational Physics from USA and 3 yrs of Postdoc in the USA. Transitioned to DS in early 2022. Mainly worked with Text data (embedding related word2vec to Transformer based, AI solutions too but Prompt based no agent based solution), Traditional ML & NeuralNets for classification and regression. Python, SQL and PySpark tech stack, AWS & snowflake platforms. Comfortable with either Linux/Unix or windows.

1. How is the job market ?

2. What are the chances of finding Job by end of my 2-3 months of severance ?

3. What should I prepare the most ? How shall I approach the job market?

Currently remote at a decent Midwest city.
Any suggestions and advice will be appreciated.

Thank you

I’m currently working at an IT company and will probably be leaving soon. I’m already talking with companies in banking, IT and consulting, mostly for roles close to my current experience.

But I also got an opportunity at a large factory with a small data science team. From the initial talks, their work seems to be around sensor data, predictive maintenance, anomaly detection, safety, maybe some computer vision. They manufacture some machines, so it sounds quite different from my usual IT environment.

Most of my recent work has been around LLMs, agents, GenAI, etc. I know this area pretty well, but I’m not sure I’m passionate about doing mostly that long term because of the hype. I still find things like gradient boosting, computer vision, time series and more traditional ML problems really interesting.

So I’m curious about people who have worked in manufacturing DS/ML. What is the culture and day-to-day work like? Is it generally calmer than IT/consulting, or does production bring its own kind of pressure? How is the work-life balance?

Career-wise, would moving into industrial ML be a risky switch in the current AI market, or could it actually be a good way to build a more specialized ML background? Also, what skills would you recommend learning for this kind of role?

Basically I have a ranking service for b2b SaaS but basically like hotels flights etc

The models do well and I can improve accuracy pretty easily to a point

But if I want to promote options better for other metrics I'm struggling to change behavior other than people selectng the same thing lower

Just hoping for experiences for those in ranking specifically and anything they might have tried other than traditional lighting ranking etc

Lately I've been deep in a personal project by writing chapter summaries of Richard McElreath’s Statistical Rethinking textbook and applying them to wildfire models, and somehow found a way to elegantly (in my opinion) combine the two through storytelling. The tl;dr: I built a whole narrative around a wildfire forensic investigator named Prof. Markov, rolling an eight-sided die to decide which direction to search a burnt forest grid, to explain how the Metropolis-Hastings algorithm (the earliest variant of Markov Chain Monte Carlo (MCMC)) actually works.

MCMC sits at the foundation of modern Bayesian computation and probabilistic programming frameworks like PyMC and Stan so it could be genuinely useful to anyone looking to level up in these topics. Roast me, tell me what you liked and didn’t like. Regardless, it was a fun little mini-project!

https://pub.towardsai.net/explaining-markov-chain-monte-carlo-using-wildfire-forensics-a334fecaefb3

I recently started working in a data scientist role for the first time, pivoting from mathematical ecology. (It's actually at an environmental organization, so the fit is great.) The job is hybrid, mostly remote. So far, it's been going really well.

Last week, they asked me to do a power analysis of a planned study. (Yay!) Of course, this requires a lot of information about measurements, expected values, outliers, what size change would be of interest, etc. I asked the necessary questions on Slack, along with some follow-ups and reminders. They were able to get me much of the information I needed and I found some in the literature, but it felt like I was bugging people (including my boss). Does anyone have communication tips on getting this kind of info from colleagues?