Facebook AI powers John Deere robotics

Using open source PyTorch to reduce herbicide usage

Precision agriculture promises improved yields using reduced inputs, made possible by machine learning models that control farm equipment and adapt to local conditions, climate, and challenges.

In this context precision depends upon accuracy, as machines are tasked to distinguish between weeds and crops. Rather than spraying an entire field, the goal is to target weeds directly, and radically reduce how much herbicides are used, lowering costs and increasing environmental sustainability.

In order to achieve this level of precision and accuracy, machine learning models have to be trained using as much data as possible. The greater the volume and diversity of data, the greater the accuracy of such models and therefore the greater the accuracy of the machine conducting the herbicide application.

This is one reason why Facebook is becoming active in the agricultural sector. As a company they build tools that can leverage massive amounts of data towards specific applications. In this case differentiating between weeds and desired plants, so that the former may be sprayed and the latter left to grow.

A great number of the AI tools currently being developed by industry leaders like Facebook are free and open source. The desire is to see these tools grow and develop as fast as possible, and by making them available for anyone to use, the hope is that they will be used and improved rapidly.

Facebook’s competitive advantage is not the AI tools they uses, but rather the vast sums of data that they possess and can use to train that data. An AI tool in isolation is useless, and requires that data to be powerful, and used for specific purposes.

Similarly John Deere’s is increasingly a data company that makes farming and heavy equipment, rather than an equipment company that happens to gather data. Although Deere faces the challenge that any company new to the data game faces: what to do with it all and how to make sense of it.

That’s where John Deere can leverage the open source AI technology developed by Facebook to make use of the data they collect from farms using their machinery.

Specifically Facebook’s PyTorch software is powering John Deere’s See & Spray robotic farming machines.

Seeking solutions to helping farmers produce more food with fewer resources, California-based Blue River Technology, a subsidiary of John Deere, has turned to artificial intelligence and robotics technology. The company’s See & Spray robotic farming machine combines machine learning (ML) and computer vision to identify weeds among crops in real time and to treat weeds while leaving crops unharmed — giving farmers a more consistent, precise, and efficient means of weeding crops.

Deere purchased Blue River Technology three years ago:

Blue River has created and fielded real-time autonomous agricultural robots. Their “see and spray” robotics platform uses computer vision to recognize plants, and sprays herbicide on weeds and fertilizer on crops with similar precision to inkjet printers. This precise targeting ultimately reduce chemical usage by more than 90%, resulting in cost savings to the farmer and fewer harmful chemicals in our soil. In cotton, Blue River’ robots can radically reduce the amount of herbicide needed to defeat resistant weeds while improving yields. With lettuce, Blue River’s robots determine how to best allocate resources to grow lettuce and autonomously thins the crop to maximize yield.

The company had been working on accurate machine learning computer vision models for at least a couple of years.

It takes time to develop and improve the accuracy of these kinds of machine learning applications, as they’d have to be done for different kinds of crops and conditions.

This announcement from Facebook is meant to celebrate the success Blue River has found, but also to emphasize Facebook’s role in the agricultural sector. Here’s more from their press release:

As the See & Spray machine moves through a field, it collects images of crops and weeds through the use of a high-resolution camera array. Each frame captured by the camera is analyzed by a PyTorch-enabled neural network to identify weeds and crops and map their locations. Once the map has been created, in a matter of milliseconds the robot then sprays only the locations where weeds were found. This approach reduces the amount of herbicide used to control weeds, passing cost savings on to farmers and promoting sustainable agricultural practices.

“This is a challenging problem because many weeds look just like crops,” Chris Padwick, Director of Computer Vision and Machine Learning at Blue River Technology, wrote in a PyTorch Medium post.

To address this, Padwick said the team at Blue River Technology consulted with professional agronomists and weed scientists on labeling weeds correctly and used PyTorch for training all of their ML models.

“We chose PyTorch because it’s very flexible and easy to debug. New team members can quickly get up to speed, and the documentation is thorough,” Padwick wrote. “The framework gives us the ability to support production model workflows and research workflows simultaneously.”

In the photo above, the green indicates the machine identifying a crop, and the red is where the machine sees a weed.

Machines are not able to learn these distinctions on their own, but require human agronomists to work with the systems and train them accordingly.

This is why the software can be free, because the data and expertise is not.

This is how Facebook has been able to build their digital monopoly, by focusing on data as the engine for their business model. Rather than selling products, they sell access to the data and the people they’ve collected.

Is John Deere working towards a similar monopoly position? Precision agriculture offers tremendous potential, but due to the reliance upon massive amounts of data, there may be limited options for farmers to take advantage of this emerging world of agricultural technology.

Certainly one of the concerns moving forward will be accuracy and efficacy. If a machine is deciding when and where to spray, then the hope or demand is that the machine is as accurate as possible.

Similarly one of the dangers of precision agriculture is using the greater accuracy to increase the intensity or concentration of herbicide applications. Historical overuse has led to weeds that are increasingly resistant to treatments. A potential consequence of smarter spraying using less inputs overall might lead to some farmers spraying more on specific spots as a means of fighting more resistant weeds.

In a previous post we discussed how Greenfield Robotics is using robotic lawnmowers to manage weeds. Their approach to precision farming is to use AI and machines to forego harmful chemicals all together. Illustrating that there are different approaches and philosophies to using technology to improve agricultural practices.

Artificial Intelligence and machine learning are finding increased roles in agriculture, but their use raises questions of accuracy and data ownership. If the tools are free, then the data is increasingly valuable, as it is the fuel for machine learning.

This reinforces the notion that an additional cash crop for farmers to consider is the data they generate as part of their agricultural operations. They may not realize that their equipment is harvesting data in addition to working their land, and that this data is being used by the equipment manufacturer to create new applications, new technology, and therefore new value.

At some point the value of that data should be shared with the farmer, or the farmer should be sharing that data with cooperatives and platforms that benefit the farmer.

Finally here’s a video of Blue River’s technology from 2018: