Recommendation for director of information science
Fast background: 25 years tech, engineering, and science expertise: have PHD w 10 years DS expertise with final 5 in mgmt. presently director of DS and DE. My IC abilities are rusty and want updating and I’ve a really younger DS and DE workforce – I really feel they want extra expertise however that I can’t present it due to lack of private expertise with the stack, Python, matillion, AWS . Is it price it to upskill so I may also help mentor my workforce, or ought to I do one thing else? fyi function approvals have dried up, so I can’t rent a employees or principal. After which there’s the entire llm AI factor occurring. On the lookout for some type of steerage right here – how finest to navigate scenario ?
Comments ( 17 )
Generally people would recommend to provide opportunities to your team members to learn.
Even if you are a fast learner I feel like you are going to stress yourself out trying to cover lot of things when finally it’s your team who will be working on it.
Is there no one in the team with certain amount of experience in whatever stack/technology you are using? Maybe try to find that out. What was the idea behind hiring so many juniors or people without much experience in the stack?
Of course you can read more high level architecture which is used for say building an ETL pipeline etc. Maybe a bit about MLOps? You don’t need to implement all the fanciest pipelines. Just so that you can plan towards it. For now just run some cron like job for any ETL stuff. Same goes for training any model etc. I would focus on doing simpler stuff but with stress on data and model quality.
Some of the topics you might want to look into is
* Do you need a data lake or dwh?
* Are batch jobs enough?
* How is data being collected? What are the issues? Quality? Automation? Monitoring?
* What are the business requirements? Like do you need to train a ML model? Or some basic heuristic solution is enough?
There are two options.
1. Upskill and guide team in technical discussions.
2. Focus more on people management and team building. Hire some good technical managers who can look at technical stuff. But set your boundaries. If you think your IC skills are rusty, please don’t try to jump too deep in any technical discussion with your teams. You will be laughed at.
Speaking from previous experience as most data science directors I have worked with were fake. They moved from engineering to data science, but faked past n years of experience as data science experience and proclaimed they know many things in data science. In just a single session, you get judged by your junior and it decides the respect you will command from your team. I have also seen many directors who know their limit and try to work accordingly by focusing on people management and team building. It all boils down to playing to your strength.
Now given that you have a PhD, you can use your skills in problem scoping, problem validation, solution prototyping, data science theory etc and guide your team and leave the work on tools like python, AWS etc. to your team. Maybe you can pick some one promising in your team, guide and fastrack them which in the long run can help the team.
Rather than up skilling yourself, how about doing something as a team? For instance, a reading group allocating working hours for reading + discussion? Or try to get some funding for training.
I’m a head of data science and understand your issue here. However I’d try and focus on what you need to achieve and then go from there to find the best route.
Firstly obviously there’s no harm in up skilling yourself, but is this the fastest/best route to what you want to achieve? Your team will learn very quickly given the opportunity, and I expect you bring a huge amount of experience regardless of stack to be able to support them.
Personally I’d focus on the best ways to up skill your team; training, projects, secondments, cross company collab? It depends a bit on what they are lacking as a team, is it coding experience, getting models out in production or business knowledge?
Obviously if you can also argue that your team needs more experience, perhaps you can go down that route.
I think you can affect more here than you’re giving yourself credit for.
My other thought in this is my personal belief that LLMs are going to fundamentally change how DS is done, so I’d focus on that more. You might find your junior team is perfectly placed to make the most of it in the near future.
Yeah, this is tough. Obviously the ideal way to address this is to have a mix of junior and more experienced ICs, but as you’ve observed, it doesn’t always work out that way. The temptation to upskill and do the technical mentorship yourself is natural, but I think it would be really easy to get bogged down, especially since you’re managing both DS and DE.
One approach that might scale better would be to give the junior folks time to research best practices in areas that align with their interests and the team’s needs. You can have them summarize their findings in a doc or present them to the team, or potentially make and support recommendations for technical decisions where you’d ideally want to ask a staff DE/DS.
Never knew directors had to actually have ds skills. Good for you to care. Mine knows the difference between excel and access and reminisces about his programming days
Don’t fake it, in my experience that always leads to junior levels not respecting the team lead and a quick change of leadership. Set your team up for success. Encourage them to go to trainings/take online courses and rely on one or more of the junior DS/DEs that show promise and help lead in discussions.
Since you are the leader, you determine what direction/deliverables the team works on and prioritizes but the implementation should be discussed and debated not dictated.
You might consider identifying people on your team with the highest personal/professional maturity and tasking them with leading an upskilling cohort for a particular topic with a subset of the team. The cohort could include those most in need of the skill and/or most interested (or if the team is small, include everyone and limit the number of simultaneous cohorts). You could lead one yourself, and also lead a sort of meta cohort with the other leaders to talk about what’s working and not working.
I think you are focusing on what you and the team don’t have, instead of fleshing out what you DO have and what can be accomplished with that.
Your team can learn – you can ensure that they learn how to learn – or at least – have a fairly solid practical 1 hour process to ensure that they have the right direction before spending a week or 2 on it.
You build a good engine, you can always accelerate later on to catch up.
It is import to stay on top of things, but as director your role is to lead the team, set directions and interface with the overall business and management, thus experience and business domain knowledge is what sets you apart from individual contributors and subject experts.
I feel like there’s proper time for managing and proper time for being a mentor. This situation calls for being a mentor. You can mold this team and make them great data scientist if you wish to.
So yeah 70% mentor 30% managing 😉
Can’t advice directly but O’Reily has several books on Data Science specifically for executives:
https://www.oreilly.com/library/view/how-to-lead/9781617298899/
https://www.oreilly.com/library/view/managing-data-science/9781838826321/
https://www.oreilly.com/library/view/leading-data-science/9781098127398/
https://www.oreilly.com/library/view/data-science-in/9781000520842/
When I became a director of a research and data science team, I tried to level myself up in all the skills that various team members had that I didn’t possess. It was not a productive use of my time. Fortunately, I realized that pretty quickly and just accepted that I wasn’t always going to be the one to come up with a clever technical solution. More importantly, I realized that it wasn’t my role to be the local expert in all the tools that my team uses.
As a manager, you only need to understand the software platform or whatever enough to help identify and/or evaluate potential solutions to roadblocks, not design or implement those solutions yourself. You also need to trust in the expertise of your hires, even if they are young and unproven. You can also serve as a conduit between your staff and others in your organization who might have the expertise to help them implement those solutions.
Explaining complex issues to a nontechnical audience is a skill that every data analyst/engineer/scientist needs to have, so don’t be shy about pushing them to explain problems to you like you’re 5. If they can’t summarize the issue and their proposed solution in 30-45 seconds, then communication is a growth area for them. Inability to communicate technical ideas to non-technical audiences is very common among those new to the field, so place some of the burden on them to help you understand the issues.
I’d also leverage ChatGPT to translate code into something you might understand better (e.g., Python or R or vice-a-versa). We have an external partner that uses SAS, which I’m not familiar with and have no desire to learn. Most of the syntax is similar enough to SQL that I can follow, but there are times when I’m stumped and rather than spend 10 minutes on Google, I just ask ChatGPT to translate it to something more familiar (as an aside, I find its SAS-to-Stata conversion to be more coherent than to open-source programs, but I digress…).
There’s a pretty good Harvard Business Review article that discusses types of managers and presents one type that resonated with me: the manager as a connector. Basically, you don’t solve your reports’ problems, you connect them to people who have the knowledge to be helpful. If you are in a large organization, then you might have that expertise on another team. If not, then you might have that knowledge somewhere in your professional network. If you have a HBR subscription, I’d recommend it as a quick and easy 5 minute read.
Who built the stack and where are they?
Yes it is, I was in a similar space with a friend and she was only a few years behind you, I’m decades behind you but aiming for your space one day.
We met twice a week for years to study code, pair program and our skills rapidly advanced since we had each as an accountability partner.
The steps depend upon your skill gaps and personal preference for skills but 100% yes it’s worth it.
If you can’t hire someone experienced, you may be able to get an agreement for consulting hrs over a year. Having someone experienced look over things like proposed tech choices, pipeline design or help with ideas to bring down cloud costs can be really helpful (I’m 4yoe and have this). An inexperienced team can do great things if they are set out on the right path with their projects initially.
You should build some familiarity with the tech stack, but having worse technical skills then your team is expected. You can’t expect to keep up with people who are programming every day.
You should be comparing yourself to bosses you have had in the past. Or to other bosses at your company. Understanding the work well is important but direct experience with the tech is only part of that.