How Omni Runs AI Engineering With Small Teams and Public Demos

amir: On this episode of the show, I have with me Chris Merrick. He is co-founder and CTO at Omni, and we're gonna be talking about, agentic within his engineering team. And we're gonna talk to him about what's different and how engineering the product teams are working. And they do something really interesting by having a demo day each Friday, and I'll have him tell us what that means 'cause they post things to YouTube.

I don't know if I've heard other engineering teams do that, so I'm really excited about how that is working. Also talk a little bit about embracing a little chaos while you're an engineering leader, and how you can handle that and some of the benefits, and But Chris, thanks for taking the time.

chris: Yeah, great to be here. Thanks for having me, Amir.

amir: Absolutely. All right. Omni, what do you guys do?

chris: Yeah, so we're, uh, an, an AI data analytics platform, so, helping folks, uh, get trusted answers from their data using AI, uh, kinda using sort of traditional modalities like dashboards and spreadsheets, and then obviously newer modalities like, uh, AI chats and agentic interfaces.

amir: Fantastic. Okay. So I've been asking engineering leaders to give us the lay of the land 'cause, um, everyone is obviously using some form of a- agentic within their development, but everyone's also a little bit different. I guess, give us the lay of the land. What, what is... I guess, how are you guys using and what tools are you using, and kinda we'll go from there.

chris: Yeah, so, um, a- around the middle of last year, I kinda said to my team, like- Uh, we are going to have to change, and I don't know exactly how this is gonna play out, but we kinda need to go and start building the muscle to start, you know, using AI as our primary development engine. and so, it ended up kind of turning into a bit of a sort of bottoms-up grassroots effort where a variety of our engineers started playing with different, uh, you know, AI coding, uh, platforms out there on the market.

long story short, kind of found the most traction with Claude Code, um, around the time that the Opus model and Claude Code kinda, uh, came together, and I think a lot of people found traction with it at that point. a-and then it sort of grew organically through the org from there. And I think it sort of, it all came, uh, kinda...

it came to completion around the holidays. It felt like kinda everybody went home for the holidays, uh, and the, let's say, you know, thirty percent of the team who wasn't quite on board yet, uh, swallowed the pill, saw the light, and came back from the holidays, uh, i-in January twenty twenty-six and said, um, "Okay, cool.

Like, this is how we do development now." and so, you know, always learning, evolving, getting better at it. but, you know, if you look at our, like, graph of, uh, commits to our main branch over time, like they sort of grow, grow, grow and accelerate around that time as well.

amir: Interesting. I, I guess, you know, when, when you are looking at the current state and you're mentioning that, uh, engineering product teams, um, and how they interact within the company, I, I guess where is that-- what are the touch points?

What do the interactions look like right now?

chris: yeah. So the interesting thing that I would say is that, um, we sort of, we were building our engineering team in a way that I think was highly compatible with AI develop- like AI development even before AI development, and then kinda got lucky that AI came in.

And, and what I mean by that is, you know, we kind of... We've had this philosophy of building, like, a small engineering team full of, like, really great people who can be, you know, highly independent in how they go pursue solving problems, uh, for, you know, customers and others. And, um, that, That philosophy has, has really, uh, you know, AI has enabled us to really kind of double down on that philosophy.

and what that means is, you know, A, we're, we're forming, you know, we've got an engineering team of about twenty-five or so engineers today. these folks work in really small groups, like one, two, maybe three people, um, go out and tackle a problem. they-- I expect the engineers to really wear the product hat in a meaningful way.

we have product managers too, and they're fantastic. but the, the goal is that the engineers should be able to be highly independent and not have to kinda go back and forth and get blocked on like, "Oh, wait, how do you want X to work?" Or, "How do you want Y to work?" you know, usually the way those things play out is like, cool, the engineer goes and builds it, and then, you know, maybe after the fact, the PM comes in and says like, "Hey, I used this for a, a couple hours, and like, I think I would do X, Y, and Z differently."

But it's not like a, "Hey, let's go write down a big document about how I want this thing to work up front." and, uh, you know, the, the other thing that, um, this, you know, that, that, that's kind of always been the case, um, but AI has enabled us to kind of push even further is like everyone on our team, we have, you know, we're fortunate to have a bunch of really fantastic engineers with many, many years of experience.

There's no job that is sort of below anyone, right? So every engineer is fixing tiny bugs all the way up to, like, building, you know, big, uh, appropriately sized for kind of their role and experience, uh, new features. And, uh, you know, our senior-most engineers are closing the most bugs. and, uh, that's kind of a, I think a, a philosophy that, you know, really resonates with me 'cause I, I wanna make sure that everybody on the team feels like, yeah, th-they're equals in terms of the types of work they're doing.

amir: were you guys running Scrum before?

Are you still running it? Have you seen any changes in terms of how you're managing and running projects at this point?

chris: Yeah, so we don't, uh, we don't mandate or sort of run a process for the entire eng team. like I said, we sort of carve out sub-projects across the team, one, two, maybe three people.

and then we sort of let those folks decide how they want to operate. one to two people tends to be almost no process whatsoever other than maybe like, "Hey, when we sign on in the morning, let's catch up on what we're doing and decide, you know, what's next and, and divvy it up appropriately." but when you get up to, you know, three people, uh, it, it tends to be maybe a little bit more process where people will say, "Okay, yeah, we're gonna do, uh, you know, uh, a sprint planning type meeting once or twice a week, um, and try to, you know, operate within like a little bit more of a predictable cycle."

but again, uh, you know, you mentioned it before, uh, I, I like to embrace a bit of chaos in terms of how we operate and, and, you know, it's not chaos for chaos' sake. but, you know, my view is always, uh, the lightest weight process that gets the job done is the right one. and so for that, for, for us, that means like, yeah, you know, if you're working on a project with another person, you absolutely need to communicate with that person about, you know, what, what each is doing, right?

and yeah, you know, it's like certainly me as the CTO, our head of product, some of the other product managers, like they're gonna wanna know like weekly sort of what you're up to, um, and sort of understand the progress and milestones. and so, you know, uh, the, the other process that we've kind of installed is a company-wide weekly demo day, right?

And this sort of serves as a bit of that check as well, where it's essentially it, it serves as a lot of, uh, it serves a lot of functions really. You know, one is like allowing engineers to sort of communicate to the entire org like, "Hey, here's what I'm working on. Here's what my priority is, and here's kind of how it's going."

And you know, I strongly encourage, you know, you're not-- your goal is not to sort of demo a perfectly neat complete feature. Like demo the work in progress. We wanna see how the sausage is being made. and people-- You, you may find that people actually are gonna give you some interesting feedback along the way that changes your trajectory slightly too so, so yeah, we, we really try to be a bit creative in terms of like, you know, where and how those processes crop up, um, to, to make sure that people are, you know, given a little bit of that freedom and autonomy to go just explore problems and solve them.

amir: this goes on YouTube, and I think, uh, for maybe for the audience, it'd be great to kinda understand the, the why and, you know, does, does everything, does it get edited?

All, all those little maybe details that go with it.

chris: Sure. Well, yeah, first and foremost, omni.co/demos. You can see every demo we've ever given. there are thousands at this point, um, s- going back all the way to, I think, uh, maybe late 2022, early... No, twen- early 2023. and yeah, this, uh, we decided to start doing this, uh, you know, less than a year after we founded the company.

And, uh- The, the idea of giving the demos was largely just a way for us to kind of like share what we're working on and kind of get people excited about like how the product was developing. But then the idea of posting them to YouTube, it, it kind of came out of this joint philosophy between myself and my two co-founders, which is that, you know, ideas are a dime a dozen, um, and that everybody's got good ideas and very few of them are unique.

but it's the execution of those ideas that really matters. and so we always said like our, our moat is not gonna be that we have some, you know, brilliant new idea that nobody else has. It's gonna be that we go out and execute that idea better and faster than, than the next person. and so, you know, that was the case from day zero.

And, and at a certain point, once we started recording these demos, we said like, "Wait, if we, if we actually believe that, let's put our money where our mouth is and actually post these on YouTube." and so we started doing that, and the impact has been pretty, pretty cool. I think one of the, the best things about it is that our customers and prospects will frequently tell us how, you know, they saw this video and they're super excited about it, and they were like, wanna, you know, go participate in the beta for that feature.

we have many customers who tell us that they have kind of a standing w- uh, weekly lunch meeting with their team to watch the videos. The, the videos get cut up and, uh, posted on Saturday, and so a lot of folks have like a standing lunch on Monday with their team to watch the videos. and then, you know, finally, it's a really great way just for the engineers on our team to kind of feel the love from the people who are using the product.

I, I always say that, you know, most engineers would rather get a high five and somebody to tell them how awesome the product is than, you know, sort of that next dollar of revenue. and, uh, this is kind of a way to really sort of connect those dots and, and make that, uh, connection come full circle. and so yeah, it's really cool.

If you watch these videos, you'll not only see the demos, but you'll see sort of the company chat that happens alongside as the demos are being presented, um, and just sort of the enthusiasm and excitement that, uh, other folks internally have for the work that we're doing and the product that we're building.

amir: That's really interesting. the first time you guys did it, um, were there thoughts, were there things that you were concerned about?

Or was it always a transparency for transparency's sake, and that's what we're gonna base the company on?

chris: Yeah. What I would say is that, um, yes, early on, um, definitely had a lot of people say like, "Oh, I don't know if you're gonna wanna post this one," or things like that. we've sort of uh, the-- we've been consistent enough in saying like, "No, we do wanna post it," that, you know, the, the, the ground rule is like, hey, you know, we can't post anything with sensitive data, you know, exposed in it.

but as long as that's not the case, we're going to post it. and, and you know, we have sort of developed a little bit of a tagging system for these demos to sort of set expectations appropriately where, you know, w-we try to sort of tag things that are like, this one's already in production, this one's a beta, this one is an experiment and maybe will never see production.

so we've tried to kinda make sure that we set expectations of the consumers a little bit. but yeah, you know, I, I think another thing that has made this successful is that You, if you actually look at the demos, well above 80% of the things that we demo are either already in production or make it there shortly after the demo.

And I think that actually helps make this like visceral and, and like really tangible for people. Like, if it was kinda all demo aware, it-- I, I don't think it would have the same impact. I don't think our customers would care about it nearly as much as they do. but because like it's, it's stuff that, you know, they actually then see in the product shortly after the demo, uh, I think they get, you know, even more excited about it.

amir: That's really cool. Right. I haven't really come across that, so I thought that was a, that was an interesting nuance. I-- And, and I guess when you, when you're looking at, you know, you mentioned the team and embracing a little bit of the chaos. Every team can, you know, uh, decide how they wanna operate to, to a certain level.

I, I guess you guys are a certain size startup at this point. Are you hoping that that's gonna hold as you guys scale and get bigger? Do you, do you feel that that might have to be something that gets, gets looked at down the road? What are your thoughts there?

chris: I view it as my job to figure out how to make sure that the important aspects of this are not lost.

Now, that doesn't mean it's gonna stay completely the same. but I, yeah, I view that as my number one job, is to make sure that the things that are allowing us to move fast, be transparent, you know, build that excitement and connection between customers and engineers, uh, a-and build, you know, quality product at the end of it, uh, I need to make sure all those things stay true.

this is working really well. I do think that, you know, like I said before, philosophy is gonna be to not grow the team at an astronomical clip. So the hope is that, you know, the team's gonna grow, but it's gonna be a little bit more methodical and incremental. so that I think will kind of allow these processes to exist maybe sort of longer than you might think.

but yeah, I, I'm not married to the process. I'm married to the outcomes, and, and I need to make sure the process evolves in the right way, uh, that, uh, allows them to continue being true for the right size team.

amir: Fantastic. I like that a lot. You mentioned you do have product managers and each person is a little bit of engineering, a little bit of product.

how does that relationship work? what's the lines of delineation do they overlap a lot, a lot more than maybe we think?

chris: Yeah. I mean, I, I think, um, the product team does still kinda maintain a set of priorities that they wanna go out and pursue.

and so yeah, I, I think it's always sort of been the case. You know, I've been in kind of an engineering leadership role for like fifteen plus years now, and, um, I, I've always sort of had this idea that like I should sort of try to match the, the type of project that I'm trying to get done with the sort of right engineer or set of engineers in the team who are not only sort of like, you know, technically capable of doing it, but also like are interested in it and insp-inspired by it.

And, and I think, um- Maybe the difference in the thing that I'm, I'm doing more so here at Omni than I had in the past is like kinda leaning into that even more. And, you know, there's absolutely cases where product says like, "Hey, nobody wants to do this project, but somebody's got to, so hey, go do it." And by the way, I usually try to raise my hand for those projects as much as I possibly can.

but, uh, you know, as much as possible, I kinda like to sort of say like, "Listen, you know, hey, engineer, like, what do you think we should do next?" Okay. how does that match up with what product and, you know, maybe I think we should do next? And, you know, is there some common ground there? Great. Then it's an obvious answer.

Let's go do it. There isn't, then cool. Let's come together and figure out like what the right answer is and, you know, what, what, which one we all believe is kind of the consensus, uh, direction. So, um, I, I think that's kind of-- Our product managers, uh, are-- We hire product managers and our product management team, they're very adaptive to the situation.

Like, they sort of understand that, you know, part of their job is to kind of nudge and inspire in addition to just sort of like setting priorities and, and, uh, you know, sort of saying like, "Okay, let's, you know, let's pull the next one off the list." Um, so there's always some, a, a process or, you know, kind of a conversation around like, cool.

Like, all right, you know, these are the six things we need to do next. Candidly, it really doesn't matter what order we do them in. They're all important. Like, you know, who's looking for the next thing that would be a particularly good fit for this project?

amir: it sounds like such a dynamic environment where a lot of people are empowered to make decisions and see out some of the details, which I think everyone always wants and they don't always get, which is really interesting.

is there a little bit of adjustment? Because I mean, a lot of places it, it may not be run that way. Is-- Do they, do they need a little bit of time? Do they just embrace it? Or how does that onboarding look for a new engineer?

chris: Yeah, so it's a really good question. a-and the short answer is yeah, I think it does. It, it sort of depends on the environment they come from. And, um, I'd say the folks that come from, you know, early startups, it's not too much of an adjustment. but the folks who come from like a Google, it's absolutely an adjustment.

And, and, you know, the thing I try to tell them, like before they start and on day one and on day two, and as many times as needed is like, "Listen, just go do stuff. Like, you, you don't need permission to go do stuff. Just go do it." and it, it, it usually, you know, it doesn't take that long to sink in typically.

but yeah, there's absolutely kind of a, a, an adjustment period. But, but again, like I think, you know, on the flip side, like it is kind of that early startup type environment that we're trying to, you know, not just-- not recreate, like just sort of retain as long as we can.

amir: That's really cool. I guess for, for-- from your perspective, you, you guys have these, you know, units of work being done.

You guys are demoing it. And I guess with a, with agentic, we're, we're trying to see some efficiencies gained. From the sp-standpoint of leveraging these agentic tools, and obviously you guys sound like you have some processes that are pretty unique, ha-have you seen or have you measured an ROI to just product, sheer productivity at this point?

'Cause I think a lot of times when I-- when we have these discussions It's really hard to measure. I think productivity is one of the hardest things to measure. You-- I think even the big consulting entities that get hired for this, it's still hard. But ha-have you seen or noticed a trend, at least in productivity for the team with Agentic?

chris: Yeah, um, it-- absolutely. So yeah, like engineering productivity, famously hard to measure. I sort of come down on like, I don't use this metric as my only input, but the one metric that I like is y- number of commits to main over the course of time. And the reason I like it is that even if you gamify that metric and try to, you know, just commit a lot of tiny little things, you're doing the thing I want you to do, which is to develop really incrementally and, and release in small chunks.

And so, uh, that's sort of the, my sort of north star metric to just understand, you know, how the team as a whole and individuals on the team are doing. And yeah, I, I, I sort of described this earlier, like as we started adopting Claude Code, um, yeah, the curve just started inflecting upwards. I actually posted on LinkedIn about last week.

w- I posted a graph of our commits to main over time, um, and it's about two to three X what it was, and that's pretty much team size, uh, neutral, uh, between kinda middle of last year, uh, even like I, I think the sort of the real acceleration really came around like September of last year, and you can just see it climb from there.

So yeah, absolutely. But at the same time, you know, I, I wouldn't say we have it all figured out, right? Like, um, definitely feel lots of bottlenecks on the code review side of the process and, and trying to kinda work through those, um, 'cause, you know, uh, I, I think this is normal, but, uh, you know, depending on what you read on Twitter, you never know that like some-- We're still in the camp that you actually have to read and understand the code, uh, before you release it.

and certainly, you know, I think there's kind of cases where that is more, more true and less true, depending on the nature of what you're building. but, um, you know, that, that has become a major bottleneck, and we've, we've done some things to try to alleviate it, and I think those have been pretty effective, but it's still, still a bottleneck.

you know, I, I think as well, like just kind of thinking through like, you know, you, you read about people like, "Oh, you know, I set up this automatic thing that like, I wake up in the morning and like the bug fixes are ready to go." we don't have that. I think partially maybe because of the code review bottleneck where it's just like, okay, cool.

Well then it's just waiting on code review, so what, what has that actually accomplished? but yeah, I, I think we're still trying to experiment with, you know, improvements to that process, improvements to our techniques, improvements to how we actually go and, and use those tools.

amir: maybe a last question for you is, uh, a- as the technology has been quickly evolving, We're also seeing a lot of, you know, talk about just the costs of using these agentic tools There's a lot of debate out there.

Is there a line in the sand in which you'd have to start looking at cost versus per person overhead, et cetera?

chris: Yeah. I, I mean, um, I think you're specifically asking about sort of like essentially the cost of tokens to go build this stuff. Yeah.

amir: The token

chris: economics-

amir: Yeah ... behind, behind all

chris: this stuff. Yeah. Yeah. I mean, certainly there is, right? Like, uh, you know, if, if you told me, "Hey, you would have to pay twenty dollars to build this feature," like most features, that's probably a good deal.

Kind of depends. But if you told me, "Hey, it's gonna cost you two thousand dollars to build this feature," like that's obviously a, a harder line to draw and, and, you know, some, some features will still make that bar, but others will not. so, uh, I do think that, you know, uh, uh, the econom-economics of this matter, right?

And, you know, they don't only matter for me, uh, and my engineering team building the features with tokens, but also like our product is a, you know, natural language, uh, you know, data exploration tool where, you know, inside of our product, we actually publish metrics about, you know, this is sort of the median cost to answer a question in the platform, uh, over the past thirty days.

And right, like, you know, twenty cents a question, usually a no-brainer, right? Like, you know, maybe, maybe there's some questions that don't quite meet that bar, but that one's pretty, you know, pretty easy. but right, like, you know, we do keep an eye on this, right? Like if, you know, two dollars to answer the question that you ask every morning, like no, that, that stops adding up.

That stops making sense at some point, right? You need to make that more efficient. If it's the same question, it should be easy, right? and, and those are the types of problems that we actually look at and think deeply about. And, you know, our, our, our entire approach to building our product is to, you know, we have this semantic layer that actually drives the cost efficiency of, um, this AI-driven, you know, data analysis down so that, you know, this question, like a question that you can ask, you know, uh, if you ask Claude Code to go like perform a data analysis, it's gonna have to do a lot of exploratory stuff to even kind of figure out like how to answer that question with the data.

Our semantic layer actually takes all that out of the question, just says like, "Cool, you know, here's a question." You can easily map it into the semantic layer, quickly generate a query and do it, you know, faster, cheaper, better, uh, than if you had done it without that semantic layer. and so yeah, we-- I think about it from both sides.

yeah, don't have an answer other than like, it's kind of one of those, you know, w- in the context of a specific, you know, feature or question, like you kind of know roughly what the bar is and everyone's sort of calibrating on that right now. And, and I think the more that we can empower, you know, as these token costs become, you know, more visceral, uh, you know, maybe you're getting subsidized less, uh, in certain cases, um- I, I think giving the users sort of control and predictability a-about that and expectation setting and kind of, you know, helping them understand, like, "Hey, you sure you wanna go spend, you know, 30 bucks to answer this?"

is gonna be an important capability.

amir: you guys are definitely ahead of the curve, which is really cool.

thanks for coming on. Thanks for sharing. I think you dove into some things that, um, I have not ha- heard, uh, other people talk about, so I think the audience will like that. But if they do have a follow-up question, uh, I'm sure I missed, uh, you know, one or two questions, um, is there a good way of connecting?

chris: yeah. LinkedIn is always a fantastic way. I check that somewhat frequently. So yeah, look me up on LinkedIn, um, and I will be eager to, uh, answer questions that anybody has.

amir: Awesome. Chris, thanks for coming on. Thanks for sharing

chris: again. Thanks, Barry.

amir: Absolutely. All right, that's it for this episode. Be back again, different guest, different topic.

Until then, two things. One, I'd love it if you could find somebody else in engineering, maybe a leader, maybe an IC, that would be of, uh, seeing this episode as value, 'cause I haven't really heard of many people that record their demos and post them. And he's-- Chris and his team are seeing some positive impact, so it'd be phenomenal if you guys could share this with other engineers out there.

And also, like, subscribe, leave me a review on Apple Podcasts, tell me if I am doing a good job or not. I'd love to hear from you. Otherwise, until next time. Thank you and goodbye.