openpipe-ai

A fully-managed fine-tuning platform for developers to create efficient, high-quality AI models tailored to specific needs.

OpenPipe AI

Hi everyone. We're at the AI Engineer World's Fair. I'm Ani Ghazaryan from neptune.ai and today I have Kyle with me from OpenPipe. 

Hi Kyle. Could you introduce yourself and tell us a bit about your background? 

Yeah so I'm Kyle. I'm one of the founders of OpenPipe. We've been working on this for the last year. We work on fine tuning, specifically moving from prompt and models to fine tuned models. 

Before that, I've been interested in working in ML for a long time. I did machine learning stuff at Y Combinator and yeah, I have a background in Computer Science. 

Thank you.

And could you tell us a bit more about the product, the components it has? 

Yeah so what we do at OpenPipe is basically usually our customers are coming to us because they've already deployed like a prompt and model. 

So they're using something like GPT-4, Claude 3 or something like that in production. 

And they're not getting where they need to be quality wise or cost wise or sometimes latency. 

That's the third one that sometimes comes up. 

If that's the case, you can actually get much better performance by moving to a smaller model. 

The problem is that if you move to a smaller model, you're going to sacrifice quality.

And so that's where fine tuning comes in. So fine tuning lets you move to a smaller model. And by specializing that model on your actual dataset and your specific use case, you get much higher quality. And so you can actually end up with better performance actually from a smaller model than a larger one if you're specializing it. So that's what OpenPipe helps with. 

There's several different components to what we do. One is the data set preparation. So we make it really easy to capture your, basically the data you're already sending to your prompt and model. And then secondarily, filter it down, duplicate it.

Have humans review it if necessary and make sure it's in really good condition so that you can then use it for fine tuning. 

And we also do evaluations afterwards to make sure your model actually works well. 

What are the common challenges that you're facing and you think will not be solved within the next 2 to 3 years? 

The challenges that we're facing as a company that we think will not be solved? 

Oh man, that's an impossible question. 

Like 2 to 3 years is like 100 years in AI time. 

That will not be solved in 2 to 3 years.

I mean I guess, one thing that we're having a really hard time automate and ultimately I don't think is fully automatable is making sure that your prompts are actually aligned with what you want. 

That's something we commonly find out as we're fine tuning the model is that the prompt does not fully capture what the user is or what the customer is trying to do. 

And so I think ultimately, basically that process of like from the user's mind extracting what do they actually want is something that's always going to involve some amount of human involvement. 

Or whoever's designing the task involvement in any case. 

Things that will not be solved.

Man, that honestly, I feel like you need some easier questions. I have too low confidence in my prediction of what the world's gonna look like three years from now. Any bets that you have, maybe, as in what will change? 

Let's see. So I think there's two really broad trends I'm excited about. Obviously the largest and the one with the greatest uncertainty is like what happens over the next three years with the frontier models. That will sort of determine to what extent all of society is completely transformed.

But even if we aren't, let's just say we pause progress for whatever reason. Where we are right now and we never got another stronger frontier model. Well what we're seeing at OpenPipe is there are so many tasks that even with the existing models that happen right now are low hanging fruit that people are building products around. 

And so I think that's already going to be hugely transformational. I think the work we're doing to make it really easy to make these models cheap.

Our product is like 25 times cheaper per token on inference compared to say, calling on GPT-4. Is going to really change the way that these products are deployed and the use cases that they make sense in and the amount of inference, the number of tasks that you can throw them at. 

Anyway we're betting on that. That basically the total usage of LLMs is going to increase by, I'd say conservatively two orders of magnitude over the next three years. I would not be surprised by 5 or 6 orders of magnitude on the upper end. 

Thank you. If there's anything else you'd like to share with our viewers, this is the time to do it. 

Yeah.

I mean if you're using LLMs. And you're doing more than a few hundred calls a day to a prompt, fine tuning can save you a ton of time. Super easy to do. Definitely check out OpenPipe. Would love to work with you on that.

Note, that there may be slight differences in text and video. Watch video for the original content.

OpenPipe AI Video