Have you looked at our jobs portal recently? Or explored openings and roles at the intersection of clean technology and data science?
Something strange has been happening over the last quarter - something that we really haven’t seen for the last couple of years. Many positions in large companies and startups that were posted late last year have been reposted again this quarter. And not just that, we’re seeing several openings posted on job sites and go unfilled despite repeated postings and connections with different networks.
So, this month, let’s go a little deeper and see what’s happening in the market.
For the last couple of years, we’ve been seeing a steady growth in roles for data scientists, modelers, program managers, C-level positions in companies in agriculture, energy, sustainability and water sectors. Initially, as we would have expected, we saw a lot of positions in newer companies - companies like Climate Corporation, Opower, Blue River Technology and so on. Most of these are startups that were later acquired by large companies in the field and the roles ranged from scientist and engineer positions to managers and C-level positions. Then, we started seeing the larger companies advertise roles and, in many cases, new teams and groups that they were building. These included billion dollar environmental consulting companies like HDR, automotive companies interested in self-driving cars and transit options, and large software companies looking to reduce their footprint and produce new products using remote sensing data, sensors and other hardware/software integrated devices.
In most cases, the roles were either posted on job sites or on more targeted groups like the American Geophysical Union (AGU) and American Chemical Society (ACS) and then were usually filled within a month or so. A typical example is the role of Program Manager, Sustainability that LinkedIn posted early this year that was filled within a month.
What’s going on now? We are seeing a wide range of roles in all types of organizations posted and remain unfilled for several months - in this case at least 4-6 months. And we’re seeing C-level roles in many VC funded startups stay open for a quarter - again, a very unusual trend, because startups need to move fast and having senior roles remain unfilled hampers their ability to deliver.
For example, Google has advertised a position for a “Technical Program Manager, Energy and Carbon” in their Sunnyvale, CA location for the last 6 months - and it’s still showing up on LinkedIn. Arable Labs, a VC funded agtech startup, has posted and reposted a “VP of Data Science” position since November last year. Blue River Technologies and Climate Corporation have been posting requisitions for Data Scientists since October - but, they may simply be growing their teams. There’s a VC funded startup in water in the San Francisco Bay Area that has been looking for a senior data scientist since October last year. And so on and so on….
So, what could this mean? It may be that companies are pulling back a little and hiring is slowing as they get picky about candidates. Of course, we were seeing this before the coronavirus scare took off - but the job market is still tight and it’s still difficult hiring data scientists, data engineers and managers for teams dealing with AI/machine learning and data science. From the general market, we’re still seeing a huge volume of unfilled positions for data scientists, machine learning experts and engineers. And salaries and compensation are still very high for these roles, so that doesn’t really seem like a possible reason.
Could it be the inability to find the right talent because of expectations that don’t quite match skill sets? When data science came into vogue, many companies were looking for the mythical unicorn - someone who could build a solid production-level code, create models and algorithms to answer questions, build beautiful visualization and have the skills to persuade executives in companies and organizations about the value of the results and how they would improve the company’s performance.
Now of course, most successful organizations that use data science look for teams - teams that consist of data scientists and modelers who build the models, dive deep into the questions and the data, data engineers who can take prototypes and scale them efficiently into production level systems, analysts and managers who can create and present visualizations of the model results and communicate the value to people inside and outside the organizations. We also now have teams in different parts of the organizations. There are data science teams in finance, data scientists in the product and engineering groups, data scientists in operations and quality, and data science teams in the business and marketing groups. And all these different roles and different teams answer different questions, need different skills and perform different functions.
So, what’s happening in the clean tech and data science world?
The single greatest challenge has been finding people who understand the clean technology sector and the problem being solved as well as what data science and machine learning are and can bring to the table. Many universities have been setting up data science programs - focused on programming, the latest algorithms and how to create prototypes. The difficulty here is that not all of these algorithms are really valuable in solving problems in the clean technology sector - because of the differences in the type and scale of data as well as the sector specific knowledge required.
Many universities have researchers in environmental and earth science programs who use machine learning and data science tools to solve problems in the specific field (water, energy, agriculture, natural resources etc.). But there aren’t that many graduates and there are challenges in going from a research focus to an engineering/product/management role in the corporate world.
And now we come to the crux of the problem. As companies and organizations have grown and been working on these problems for a while, they have come to realize the need for teams and experts who can understand the clean tech sector, data science and translate these to the people in charge. In many ways, it’s similar to the transition that happened in the broader data science community when companies stopped looking for someone with all the skills and started looking for teams and ways for people in their organizations to learn new skills quickly.
And so, in many cases, they're clarifying the role and responsibilities as they interview people and then reposting the job with a clearer focus. Or they're realizing that they need a larger team than they had envisioned at first and are looking for initial hires who can bring as broad a skillset as possible. In the case of large clean technology firms, they're interested in people who understand the sector and where data science can be used - and that is a relatively small section of the job seeking population. So, jobs get reposted and stay unfilled as people in clean technology sectors look for ways to add skills and data scientists try to understand what they can bring to the sector.
What do Google, Climate Corporation, early stage startups in farm robotics, and researchers trying to figure out how to feed the world sustainably have in common? They’re all grappling with one of the toughest challenges of working with natural systems - how do you work with data that is sparse, unevenly distributed and with systems that have so many connections and interactions with other systems? Before the advent of cheap sensors that are connected to phones, easily accessible satellite data and drones that can fly over fields quickly and inexpensively - scientists in companies and academia worked on developing plant and crop models that incorporated as many aspects of the farm and as much data as was available so that they could understand and predict what was likely to happen on the field. Understandably, the forecasts took some time to produce and as the models grew more complex, issues about how to estimate model parameters and the uncertainty associated with the resul
A mid-sized data center consumes around 300,000 gallons of water a day, or about as much as 1,000 U.S. households; About 20% of data centers in the United States already rely on watersheds that are under moderate to high stress from drought and other factors; Operating a data center often requires a tradeoff between water use and energy use; And in a survey of 122 data centers in the United States, only 16% or 20 utilities reported plans for managing water-related risks. As professionals working in the field, what can we do to solve this issue? One aspect is developing and using water models that can identify water risks at different scales - so that we can predict the risk to water supplies under a changing climate. A second is using machine learning to identify and optimize water use between all the stakeholders in the watershed - data centers, farmers, cities, other industries - so that biases and needs are brought out into the open and the key issues identified. A third, of cours
Our online community space is now open to anyone who has signed up for a free or paid course on our website! In addition to everyone who signed up for our cohort-based courses, we're now expanding it to all the members of our community. If you've already signed up for any of our courses, check your email for the invitation for the space. It's where we'll get together to talk about all things data science and clean technology related, discuss the latest research, network and make connections with other professionals in the sector. It's an invitation only , no bots and no trolls allowed space - so come on over! Here's where you can check out our courses and join our community !