r/sysadmin • u/Significant_Sky1471 • 1d ago
Rant I understand the concept. executing it under pressure is a completely different skill
I can explain how DNS propagation works and probably i can diagram it. what I cannot do is troubleshoot it in real time while three people are staring at a shared screen waiting for me to just fix it already.
senior fixed a ticket in the time it took me to finish reading it and ofcouse it's not because I'm slow but he's seen this exact failure pattern 200 times and I've seen it like twice? that's not a skill gap, that's a reps gap and when you finally get the girl job nobody warns you how loud that gap sounds in a room full of people who've been doing this since dial-up
got told "trust your instincts" by my lead. cool! Great advice! my instincts currently have a sample size of six months and mostly say "panic, then check Reddit"
the worst part is getting asked something in standup and having that half-second where I'm trying to figure out whether I know the answer or I'm about to bullshit everyone (don't push the red button!)
anyway I got a ticket resolved today without asking anyone for help and I'm riding that high for the rest of the week. small wins. some of the senior guys probably don't remember what this stage felt like anymore, but I will and I'm taking notes
56
u/7twenty8 1d ago edited 1d ago
The real difference between senior and junior people is panic. More senior people just don’t give a fuck because they trust themselves to figure things out. More junior people panic. Our memory systems don’t work properly when we panic so it’s like trying to reindex a database when the server it’s on is on fire. Quite literally in this case.
Panic is a bad thing when you deal with systems. The panic system was specced and resourced when we had different problems - like big fucking animals coming into our homes and eating our entire families. Our brains didn’t change dramatically between being hunter gatherers and being sysadmins or software developers. The physical and mental changes necessary to keep a tiger from eating grandma are a lot different than the ones we need to keep systems up. When you panic you go into that primal place where your short to long term memory retrieval and encoding process works differently.
Avoid panic… avoid the fight or flight stuff that doesn’t help you and this stuff will resolve itself. When you no longer panic, it will be hard to explain this to people who do so it will seem like you don’t remember.
5
u/Significant_Sky1471 1d ago
makes sense. half my problem right now is panicking before I even start troubleshooting lol
working on that too
•
u/YachtingChristopher Jack of All Trades 23h ago
This is the answer. Learn everything you can about as much as you can. Have a home lab. Break stuff, then fix it. Nothing prevents panic like knowledge. You can learn more in the next 12 months this way then 5 years at work.
•
u/Significant_Sky1471 16h ago
yep! that's the idea. my home lab is a field of fails and loss haha
•
u/7twenty8 13h ago
If you panic first, you’re gaining nothing from a home lab. You would be significantly better off if you stopped working at home entirely. Or if you do only work on things you’re good at. Retire it for now or make it a lab of success. You can’t encode or retrieve the right things when you panic. Time off away would be more beneficial to you than failing.
28
u/delightfulsorrow 1d ago
anyway I got a ticket resolved today without asking anyone for help and I'm riding that high for the rest of the week. small wins.
Well done, enjoy it. You earned it.
some of the senior guys probably don't remember what this stage felt like anymore,
Probably. But enough still do and are happy for a junior getting there, too.
Wait for the first time one of the greybeards popping up at your desk asking you for help because you are deeper into that specific topic than they are. That will be even better.
I'm 35+ years in the industry and currently mentoring two juniors who, after half a year, slowly reach that point. It's fun to watch how they are growing.
3
u/Significant_Sky1471 1d ago
okay this genuinely made my day hahah
thank you for support! i'll remeber comment when time will come (can't wait for it)
•
u/Broccoli_Ultra Sysadmin 12h ago
Hey I'm a year or so ahead of you with experience and what the poster above said has started happening to me. It was a real mentality shift the first time it happened, to get someone with decades of experience popping a question. Sometimes I think they come to me because they feel a little embarassed to ask each other, and thats fine!
I am also a panicker/self-doubter and the only advice I can give you there is exposure therapy. You'll chill a bit in time as you recognise your skills growing. I could barely type when people were watching me not so long ago.
A lot of people in this industry seem to hang their self-respect on their skills at work - so you may come to notice there's more doubters than you think around you, or people trying to cover the fact they don't know something. Basically - you aren't alone and this is normal!
5
u/ddBuddha 1d ago
You might be making a joke but if your instincts are telling you to panic you legit need to work on that. Being able to be level headed during an emergency is an important skill in this field, and acting while in panic mode is how mistakes happen. Learn to take a breath, take a step back and evaluate the situation so you can determine the appropriate course of action.
3
6
u/BrainWaveCC Jack of All Trades 1d ago
the worst part is getting asked something in standup and having that half-second where I'm trying to figure out whether I know the answer or I'm about to bullshit everyone (don't push the red button!)
It always takes time. It's just that the amount of time it takes goes down as your experience goes up.
Don't feel like you have to solve things before you look at them, and don't feel like you have to pay attention to all the eyes on you in the moment.
Yes, even though it's been a few decades since was in the same spot, I remember how it felt. And that's why I am tell you that you just have to not care about it in the moment.
I once had to tell a Director, who kept hounding me every 10 minutes for a status, that he could either let me get a good 30 minutes to assess and fix it, or he could just talk about the problem every 5-10 minutes for a much longer outage window. He finally left me alone, and I was able to initiate the fix in 40 minutes, with the issue being fully under control in about an hour. And then I had useful info for the debrief.
You'll get there. Don't stress it. This is just how it goes in the learning and growing process.
•
9
u/wunderhero 1d ago
The real skill you learn with time is the ability to NOT panic. Push it down and keep focus on the problem.
1
u/Significant_Sky1471 1d ago
that's wise advice though all comments here. like it and will take it with a little courage for my next working days hahah
9
u/BoltActionRifleman 1d ago
Any time I make DNS changes I intentionally work at the pace of a sloth. If someone doesn’t like it, I don’t care, it’s my ass on the line, not theirs.
8
5
•
u/Bright_Arm8782 Cloud Engineer 21h ago
I'm in the game over 20 years, I permit myself 30 seconds of "How the fuck am I supposed to know?!"
Then I get together the information that will let me sort the problem out.
4
u/kraftinfosec 1d ago
I've been in IT for around 20 years and I still have moments like those. Someone calls you up about a system you don't really know and there's some error message. Your first instinct is "I don't know this system, I don't have enough information, if only I had all the information then maybe I could fix the problem." You soon realize this is true about all systems. Few people know exactly how every system works, even if you're pretty familiar with that system. You can still solve problems without ALL of the information.
What you then realize is that there are some basics in IT that seldom change:
- Users need access via accounts
- User accounts are tied to authorization
- Unless you're logging in directly to a terminal, there is also likely network involved
- It's ALWAYS DNS
- If it's not DNS it's probably TLS, but it's also still somehow related to DNS
- Users lie about everything, whether intentionally or not - Get a screenshot of the error message or reproduce it yourself
Just start with the basics and build the system up in your head. Is it powered on? Is it connected to the network? Can I ping it? Is the port open? etc. Also, always ask someone familiar with the matter what changed, at least three times. Space it out so it isn't obvious.
That stuff does take time to learn, but it is the fundamentals. It's table stakes for doing System and Network Administration. Just slow down and take your time, and you will figure it out.
5
u/f7063 1d ago
Don't trust assumptions about the problem, check again. It's easy to get tunnel vision from others or yourself when analyzing. Careful to not be to annoying doing this aswell lmao
5
3
u/Vektor0 IT Manager 1d ago
AI slop.
3
•
3
u/OneSeaworthiness7768 1d ago edited 1d ago
On top of the AI written post, let’s say it was a real scenario… like did they somehow become a sysadmin that’s never resolved an issue or done any kind of support before? Bizarre.
2
u/Asleep_Spray274 1d ago
That's the difference between theory and experience
0
2
u/f7063 1d ago edited 1d ago
I know a lot of people that panic when sharing the screen. Not to say that i never panic. I do think it's sometimes about who is there aswell. While it is unconfortable I do think these problems kinda get easier with time, usually we get someone in a call with shareholders to get the beating, usually your manager, while the others work on the problem.
You did narrow it down, the more you've seen these problems in the wild the more you are likely to spot them it's just pattern recognition. Why do you know what is broken? Either you understand and have the analytical capability of drilling it down or you set this whole infra or some kind of monitoring knowledge of specific problems.
Either way, dont burn out just because someone that is more senior than you know more or is faster, that's the whole point of having someone with more experience. Just try to keep your cool and do it on your own pace, ask and keep errors to the minimal as possible. Don't let fear freeze you but don't go wild.
Good luck EDIT: some typos
2
1
u/Significant_Sky1471 1d ago
thank you so much! i'm sure that further steps will be easier and wisier but right now this reddit comments really cheered me up
2
u/dcaponegro 1d ago
To be fair to you, if the senior tech has seen an issue pop up 200 times, they are not much of a senior tech. Issue should have been fixed after the third failure.
2
u/RegardedAsshole 1d ago
You sound like a funny coworker to work with. Practice seeing many issues and asking many questions. The panic thing will become less relevant over time. I have the same issue but once your confidence and experience grows, your mentality will shift from "omg i have no clue what to do" to "Well lets see what we can do here".
•
2
u/jcpham 1d ago
Command prompt or a terminal. 99% of problems can be diagnosed from a handful of commands. If people are looking over my shoulder they get the terminal and me typing commands that I’m not bothering to explain.
- can’t ping it
- name doesn’t resolve
- no gateway/route to host
- blah
1
u/BatemansChainsaw 1d ago
three people are staring at a shared screen waiting for me to just fix it already
why the hell is anyone looking over your shoulder? that's absolutely barbaric.
•
u/Unable-Entrance3110 14h ago
My self-censor routine goes into berserk mode when I am watched. It causes a lot of false starts and mistakes. Makes me look like a real imbecile. Which then feeds back and adds more noise to the self censor routine. It's a real downward spiral...
•
u/idspispopd_iddqd 8h ago
How are the top two comments from u/SamOakTree and u/7twenty8 most voted on? LMAO. Hey, I have depression do you think you could help me out somehow? Oh, just stop being depressed? I guess I never thought of just not doing that!
u/significant_Sky1471 You have done the hard part already - you recognize you have an issue that you want to shore up and be better at. Everyone deals with anxiety and stress in different ways and it doesn't say anything about you one way or the other if you can naturally deal with it or have to work on it a little. We aren't all able to experience these situations because of how different our lives are.
For me it was public speaking. I knew it made me nervous as all hell and I couldn't seem to get a handle on it. I forced myself to make progress by speaking up more in meetings, placing myself in situations where I need to lead in front of others, and any other time I could speak up. Repetitions - you already nailed it. You just need to experience it enough so that it becomes normal and not panic inducing. Put yourself in those situations where you have to drive but do it smart, if you can start slow and small and work your way up to larger groups. Eventually you will realize its like being at the gym - no one is paying attention to anything but what they are doing or are responsible for, just like meetings. No one is making fun of you or think less of you, either they all know what the answer is and are not paying attention or they don't know and they are trying to learn as well. IDK, just my .02c - just telling someone to 'stop feeling that way' is not helpful or productive.
•
•
u/EmuLongjumping4779 8h ago
I wouldn’t say trust your instincts but rather be confident in your actions. We have all been there and continue to be there even after 2 decades. Also never be afraid to ask for help!
•
u/vogelke 3h ago
I was stupid enough to treat something non-critical as a crisis, transposed two steps, and trashed several weeks of production files by overwriting them with backups. It was the only time in 15 years that our helpdesk/IT shop got a does-not-meet-standards rating.
If you're not dealing with fires, floods, earthquakes or gunshot wounds, just follow your (hopefully) written notes and don't get random.
0
u/weaver_of_cloth 1d ago
I have been a sysadmin for 30+ years now and I STILL get anxiety attacks when someone is watching my screen. Even if I 100% know what I'm doing. Even over Zoom. Some things are just not OK.
0
u/OptimalCynic 1d ago
We've all been there. Most remember what it's like and won't ride you for it. Some don't remember and don't have enough empathy to work it out - those people are hell to work for. This is, unfortunately, part of life in an industrial society.
It's like when you see someone while driving with learners plates. Decent people give them plenty of space, don't expect too much, generally make allowances. Wankers will tailgate them, honk at them for not responding instantly, etc. You've got your L plates on at the moment - that'll change with time.
edit: And always remember step 1 - take a deep breath
•
u/graffix01 23h ago
i find it helps to refra me it in your mind. imagine someone else is having this problem and everyone is looking at them. what would you advise them to do?
171
u/SamOakTree 1d ago
You have to just not care.
At one job I worked very closely with the dev team so they pulled me into this group chat because the dev team couldn't get to a certain resource.
Everybody was talking sideways and his VP kept saying the problem is X but he had no proof that the problem was x. So I said everyone let's just stop guessing and get our baseline. Give me 10 minutes to talk to the development manager and then we'll know which way to go.
The VP kept saying no no this is the issue and I said we don't know that for sure let's find out. And he just abruptly left the chat.
So I talked to the development manager and it turns out the issue isn't what they thought it was it was that they couldn't get to a specific resource. I knew they had made some changes in Azure so I looked at the logs and they had blocked an IP address.
I spoke to that team and told everyone in the chat to wait while I spoke to the team. And I asked them to whitelist that IP address. And it solved the issue.
You have to just take charge and just trust yourself. Being confident and cool headed in those situations just comes with time. You eventually get to where you realize it's not all a big deal so you don't need to panic.