Reality Capture LIVE: Reality of Robotics

If you've ever thought about using robots on your project, you've come to the right place. In our recent Reality Capture LIVE session, we got the opportunity to interview Chris McKee from Turner, Brian Ringley from Boston Dynamics and Haydn Bradfield from DroneDeploy about all things robotics.

In this blog, we captured the highlights from the session so that you can dispel any misconceptions you may have about robotics and help reveal a crawl, walk, run approach to bringing robots to your next project. Let's get into it.

Watch the full conversation

To start off tell us a little about each of your backgrounds

Chris McKee: So I've been with Turner for five years. I've been surveying for about 30 years, and I'm very curious about just everything regarding construction tech. So it didn't take long for me to get involved with robotics and laser scanning. I really got involved with it when our headquarters in New York got Spot from Boston Dynamics and started testing it, and I immediately started lobbying for us.

I'm like, we can use that on the site–I know we can. So we got it and we took it to a whole new level. But it took a lot of work. We had to find the right combination of software to people. Fortunately with Haydn and DroneDeploy–that was a huge link that was missing, that really helped out.

Brian Ringley: I come out of the architecture and construction industry. I was actually one of the very first paying North American customers for Spot and tried out doing a lot of the things that Chris is successfully doing today.

In 2019, toward the end of the year, I joined Boston Dynamics to focus on the applications of this robot and potential solutions in the construction space. That eventually included mapping and reality capture at large. 

Haydn Bradfield: For DroneDeploy, I'm a product specialist for the automation and robotics team. But I actually started in the construction space, so civil engineering large infrastructure projects– hands on construction.

Like Chris mentioned, I developed a real interest in the geospatial components. And then sort of slipped into doing a lot of surveying which obviously evolved into surveying and reality capture. I then decided to take a break from reality capture and moved on to work for one of the larger distributors of reality capture.

It was kind of like this hybrid of experience and skillset that put me into a position here at the DroneDeploy robotics team. My day-to-day goal is to help customers like Chris actually onboard robots, understand how all these technologies can come together.

How would you describe this idea of robotics to other people?

Chris McKee: When we have to go to different job sites, we ask them what tasks you have that are repeatable. What is it that you do over and over and over again? Let's say you take pictures that get uploaded to a platform for regular viewing and virtual inspections.

Well, if you've got someone out there doing it, going to the same spot over and over again that’s where robotics comes in as an excellent solution. Why go into a space and do a scan repeatability when we can automate that through robotics. Like we have rooms that we go into that we do stages of scans before walls go up and then after the walls go up, and then when a ceiling goes on, and then a final scan–that can all be done with robotics.

Anything that deals with just that constant repeatability can be solved with robotics.

Brian Ringley: There's definitely a collection of new technologies that when we figure out how to use them in collaboration with, with human intelligence allow us to do things both creatively and in business, that just wouldn't be possible with just people alone or with these technologies alone. This new class of agile mobile robots that can move around the world and manipulate the world and interact with people, that is something that's new.

So as a company, our mission is really to show all of the positive uses of that and how that actually enriches our lives. Because it takes some getting used to. I think it's one of the hardest things about making this technology useful is getting past the distraction of how novel it is.

Haydn Bradfield: I think as we've evolved over the last two years at least, we are tending more towards automation. So if a customer or a particular industry has a thing they want to do, how do we enable them with a set of technologies and a workflow that can achieve that in a semi or fully autonomous way to make it either less painful or more efficient for the business.

So yeah, those three D words come into play here, in terms of getting rid of the dull, dangerous and dirty work. I think that's probably splattered on every single robotics page out there that someone is selling. But I mean, that's obvious, really. Who wants to spend the time and effort manually mapping a solar farm when a robot could do the job autonomously.

And a lot of the times, the conditions these jobs are done in aren't meant for humans, so why not put a robot there that an operator could either drive or it's collecting information on its own. Or have six of these robots spread across six different sites and have one person sitting cozy at home, being able to do the job of six different people and not even break a sweat doing it.

So I think we are tending towards autonomy rather than trying to identify as robot handlers, if that makes sense. 

Chris McKee: Something to add to that too, is that you’ve got students that are coming out of universities that are paid a whole lot of money for their degrees just to go and take pictures on a site. And the question we have to ask is would someone with a four year degree in engineering want to take pictures all day.  

So for us it was like, we can take that person and we can move them into a position where we can start training them and use automation and Spot to do those tasks.

What are some of the misconceptions you are all hearing around robotics?

Brian Ringley: One major one for me is the fact that a robot doesn't have dependencies on external inputs, whether it be GPS for navigation or some other kind of network. I remember early on, in fact, when we were first working with Turner in New York, we were talking about how a lot of job sites are connected in the sense that they might have like a cradle point or two, but they tend to be clustered by where people have desks.

At first for safety, we're like the robot always has to be connected to a controller, but if you're running it autonomously maybe you don't care and maybe you don't need to see what it's doing and you can just build that trust in the machine. 

So we actually took that away kind of at the request of Turner and other construction customers in particular, just saying we at Boston Dynamics can't maintain that connection remotely, just let it run truly autonomously where it's got its own mission internal to the robot and it's able to execute that without you being there. And then they're like, well, what if I need to stop.

It's like, well, you, you can't stop it because now it's not connected to anything. So it ends up being a trust building exercise.

It’s like, I can run this thing at night but do I actually trust it. So once you make it practical to get the ROI, you're just beginning a trust exercise with it. 

I don’t know if that's a misconception, but rather it is just about something that I think can be overlooked when people are planning on doing these types of technology pilots.

I think another misconception is that it's all about mobility. A lot of conversations are like, can this robot really move through these spaces? Can it really go up and down stairs? Can it really avoid obstacles? These are legitimate questions, but this is our bread and butter. We will solve all of those things, that is not the question you should be asking. The question you should be asking is, so what's so valuable about scanning every day? What do I do with 10 billion points that I collected last week?

Just because I can do that now, should I? So I think that the misconception about robots is focus. Should the focus be about robotic technology, no, it should still be about solving your business problems. The robot needs to be in service to the business problem. The robot, in and of itself does not have inherent value. 

Haydn Bradfield: Point clouds are not synonymous with progress tracking. Just because I’ve done a point cloud scan doesn’t mean that all of a sudden the robot must be able to create a model and tell me how much progress is done or what's changed. And it’s like, no, the robot's able to collect point cloud data and deliver it in the right format to your VDC team, who should have a workflow already running that can accept process, put in Navisworks, do their checks, 

And testament to Turner, as they've done that, which is why we were successful as all we did is augment their current workflow with a more efficient way of collecting data.

If I can all of a sudden get a few billion points a week from an autonomous vehicle, what am I gonna ask is, how valuable is that? What is it gonna mean at the end of the day? And then the second one is, yes, the robot has got a lidar on it, but that lidar isn't necessarily made for the mapping that we know and love from terrestrial scanners.

So sometimes people say, oh, I'm buying the lidar unit for the robot, so it should be able to generate this beautiful point cloud that I can use in Navisworks.

Unfortunately, that's probably not the case. So we really try to teach the industry on what to expect and to manage expectations. 

Chris McKee: I get a lot of, how much is that ai? Is that AI that's in it? No, it's not. It's not gonna take over the world. It's not gonna come and do bad things on site. 

But we also have to manage expectations on site too. Once it’s set up people will eagerly ask us, can we do a point cloud now, does that tell us what we're off? Well, we have to compare it to a model. That's usually the big thing that we get is we have to compare it to a model and that takes time and there's a process.

So, Chris, how did you approach the crawl, walk, run method at Turner?

Chris McKee: So when I first arrived at Turner New York, they were very much in the crawl phase. 

They wanted to see if robotics were even capable. Can it walk? Can it walk over rebar? Can it walk over a pallet? Can it walk over debris? Can it go through water, you know things like that. So it was just this sheer mobility question of…can I get it around a jobsite because a jobsite is constantly changing. And the only way we could find that out is if we put it on an actual jobsite. And we found very quickly that, yeah, it works.

So after that we moved on and said, now let's try to make this autonomous. That's when things got a little more challenging because the jobsite's always changing. For example, we have these portable plastic stairs that you use to get in and out of areas. 

Well, some trade partners would leave these stairs in the middle of the floor for the next day. So Spot was walking and walked right up the steps and right off the other side and crashed. So that was a learning experience for the autonomy side of it. It's like, well, okay, if we have a map, how do we tell it that there are stairs there?

Now if it's in the way, now what do I do? Do I find a different path? And that's where the map comes into play. Once we got that down, and had a map, we ran. We started to run missions, and our trade partners were were coming in with these huge platforms of pieces of equipment and set it right down in the middle of the path. And thanks to our map Spot was able to avoid it. 

It got to that point where we really had to know our site as we had to create regular maps to get that autonomy.

And then once we got that then it was just, okay, let's just add stuff on it. Let's add a scanner. Let's add a camera. Let's put on a, a sensor of some sort and just let it run. So once we got the autonomy piece down, everything else fell into place. 

Brian, I’m curious, what do you hear most from people who are just starting to get interested in robotics. 

Brian Ringley: Turner has a long history with us. They were one of our very first customers in 2020 during our early adopter program.

And so I think like what Chris was talking about, it was a different era too, where you had to build trust in these basic layers of mobility and autonomy and payloads. But now, there's an ecosystem out there, right? The work that DroneDeploy has done with Turner has made this all commercially available.

Like you can ideally just start to run because, you know, we've built an ecosystem with our partners in this industry. But I still think, first and foremost, because we have that, and because you don't have to worry about your R and D  group doing all of this  extraneous testing with the robot, it becomes all the more important that you can actually isolate what's valuable about this robot.

These solutions are really revolving around automated reality capture. Like what are you doing with reality capture today that can really help accelerate or even fundamentally change the way you work? So I think that's always the first question I get. Like, don't worry so much about whether the robot can walk your site.

I assure you that that is not the hardest problem. The technology is not the hardest problem. The hardest problem is being able to clearly say what the value is for the business problems. And we stay humble and we learn from customers who want to use this stuff to improve how they're practicing.

So we really want and need to hear those things to keep advancing our partnerships and to keep advancing not just the robotic technology, but the whole end-to-end solution and ecosystem.

And Haydn, what do you find yourself saying to folks who are wanting to adopt this technology.

Haydn Bradfield: Well, we try and bring realistic expectations as to what's available right now.

But then we look at the bigger picture and we say, all right, we have a vision. This vision is that we want the robot, this is the job that the robot needs to do. So then we can obviously, you know, break that up into what we need the robot to do day to day, or component by component to get that done. If you are a customer who's thinking…I can just pick something up off the shelf, and I have this research team and I'm gonna break up my dream into these pieces of work. And then every time I'm gonna go and do a piece of work, I'm gonna go to a robotic vendor off the shelf and try and prove this concept…That limits people in terms of achieving that overall dream. Whereas the value of partnership is like, all right, we are in the horse race, we are choosing a horse, we are backing that horse, let's go for it.

And you've obviously gotta choose the right horse. But we've put ourselves in such a position where we value all those partnerships, where we have a mining customer who wants to achieve this amazing thing in five years time. That's brilliant because that allows us to line up for the next five years or the pieces that we need to do to solve that industry problem.

And it's not a problem that can be solved overnight or in three months. It's literally a five year strategy with the right partnerships over those five years with the right robots, the right sensors, the right people to build that dream. So we're always trying to help people on that level think big.

‍

‍