Heiki Riesenkampf is from a tiny country called Estonia, later studying computer science in London and doing his post grad in Zurich. He has been into machine learning since before all of the hype it has now. Outside of technology, he dreams of being a macroeconomist, and spends a ton of time reading about the topic. He lives in New York now, and frequently takes in the architecture, fashion and local art scene.Previously, Heiki spent time working for a VC, eventually building a product in a completely different domain. After personally realizing that he didn't want to be known for the product he was building, he pivoted towards building something that impacted him personally as an immigrant.This is the creation story of Commonbase.SponsorsP0 SecuritySpeakeasyQA WolfSnapTradeLinkshttps://commonbase.com/https://www.linkedin.com/in/heikirOur Sponsors:* Check out Vanta and use my code CODESTORY for a great deal: https://www.vanta.comSupport this podcast at — https://redcircle.com/code-story/donationsAdvertising Inquiries: https://redcircle.com/brandsPrivacy & Opt-Out: https://redcircle.com/privacy
This episode is sponsored by P0 Security. Cloud governance is a problem facing many organizations. With P0's universal access governance platform, your security team can identify access risks and automate the user access lifecycle, all without interrupting developer productivity or disrupting production operations. Visit p0.dev to learn more and secure access for all identities, human and machine.
People ask me, why did I go into machine learning when I was able to do research in any part of computer science? I personally always felt that machine learning was the one part of computer science where if you showed someone what state of the art was able to do, an average person that didn't know how it worked, they would always feel like it's magical.
I feel like that magic that machine learning has, it's still there. That's the reason why I love that space so much. That's the reason why I've stayed in that space so much. That's the same principle that I apply to the products that I build. My name is Hege Riesenkamp, and I'm the founder and CEO of CommonBase.
This is CodeStory. A podcast bringing you interviews with tech visionaries. Who share what it takes to change an industry. Who built the teams that have their back. Keeping scalability top of mind. All that infrastructure was a pain. Yes, we've been fighting it as we grow. Total waste of time. The stories you don't read in the headlines. It's not an easy thing to achieve.
Took it off the shelf and dusted it off and tried to begin. To ride the ups and downs of the startup life. You need to really want it. It's not just about technology. All this and more on Code Story. I'm your host, Noah Labhart. And today, Hauheiki ReasonConf has made a way for you to talk to your customers by translating your speech to their language. This episode is sponsored by Speakeasy.
Grow your API user adoption and improve engineering velocity with friction-free integration experiences. With Speakeasy's platform, you can now automatically generate SDKs in 10 languages and Terraform providers in minutes. Visit speakeasy.com slash codestory and generate your first SDK for free. This message is sponsored by QA Wolf.
QA Wolf gets engineering teams to 80% automated end-to-end test coverage and helps them ship five times faster by reducing QA cycles from hours to minutes. With over 100 five-star reviews on G2 and customer testimonials from SalesLoft, Grotta, and Autotrader, you're in good hands. Join the Wolfpack at QAwolf.com.
Hagee Riesenkamp is from a tiny country called Estonia, later studying computer science in London and doing his post-grad in Zurich. He has been into machine learning since before all of the hype it has now. Outside of technology, he dreams of being a macro-economist and spends a ton of time reading about the topic.
He lives in New York now and frequently takes in the architecture, fashion, and local art scene. Previously, Heike spent time working for a VC, eventually building a product in a completely different domain. After personally realizing that he didn't want to be known for the product he was building, he pivoted towards building something that impacted him personally as an immigrant.
This is the creation story of CommonBase.
CommonBase today is a universal speech translator. We help people speak any language in real time and help people connect with each other across language barrier and also cultural barriers. The company was started last year. After having spent some time working on the initial product, I realized I didn't want that to be my life's work.
That's after having moved to the US, I realized that my unique lens, my unique personal experience, led me to discover the language problem, especially being here as an immigrant and realize that, hey, this is something that I have personally felt throughout my life needing to adjust to different languages and cultures. And this is something that I want to see as my life's work.
And so that's the story of how I got to what I'm currently working on.
So tell me about and this will this will be super interesting. Tell me about the MVP for CommonBase. And this is the this is the second version. Obviously, there's going to be technological overlap like we talked about. But the MVP for what CommonBase is now, how long did it take you to build and what sort of tools were you using to bring it to life?
The very first version of CommonBase was built on top of third party APIs. If you think of the problem of speech translation, you have different ways of solving the problem, but at a high level, you need to take the existing speech, you need to transcribe it into text, you need to take the text, translate the text into another language, and turn the translated text into speech again.
So that is the initial pipeline of how we built the first version of the product. And the different tools that we relied upon was Twilio for connecting phone calls. And we ended up going with DeepRAM for the speech transcription. Now we're using Llama run on Grok for the translation, and we're using 11 Labs for the speech generation.
So that was the stack for the initial MVP that was just to prove the point that real-time speech translation at a very simple level is possible in 2024.
Let's stay on that MVP for a minute. With any MVP, any version of a product, you have to make certain decisions and trade-offs around how you're going to approach it. And I hear you saying you approached it to prove the point. Tell me about some of those decisions you had to make. Feature cut or approach or tech debt acceptance, all those things. And how you cope with those decisions.
I built the product... In a developer environment that I had never used before, I used Replit in order to build the very first version. And Replit was always known as a kind of education tool. What I liked about Replit, how easy it was to invite other people to basically check out your code and basically help you reason about the code. And so that was an interesting tool.
I'm pretty sure that we will grow out of Replit. I just want it to be as scrappy as possible and as collaborative as possible. And so that was an interesting choice that I do not regret making. And probably on the photo stack part, I decided to use Twilio instead of basically building a web-based application because a regular phone call is such a standard format for...
Initiating an audio call, I would say looking back at it, I probably would have preferred to build it on top of like two different browser sessions instead of complicating things with bringing the phone connection into it.
This message is sponsored by SnapTrade. Link end-user brokerage accounts and build world-class investing experiences with SnapTrade's unified brokerage API. With over $12 billion in connected assets and over 300,000 connected accounts, SnapTrade's API quality and developer experience are second to none. SnapTrade is SOC 2 certified and uses industry-leading security practices.
Developers can use the company's official client SDKs to build investing experiences in minutes without the limitations of traditional aggregators. Get started for free today by visiting snaptrade.com slash codestory. This episode is sponsored by Speakeasy.
Whether you're growing the user adoption of your public API or streamlining internal development, SDKs can turn the chore of API integration into effortless implementation. Unburden your API users from guessing their way around your API while keeping your team focused on your product. Shorten the time to live integration and provide a delightful experience for your customers.
With Speakeasy's platform, you can now automatically generate up-to-date, robust, idiomatic SDKs in 10 languages and Terraform providers in just a matter of minutes. SDKs are feature-rich with type safety, auto retries, and pagination. Everything you need to give your API the developer experience it deserves. Deliver a premium API experience without the premium price tag.
Visit speakeasy.com slash codestory to get started and generate your first SDK for free. So you've got that version. You've proved the point. And I know it's still early days, but how are you maturing and progressing the product from there? And I think I'm curious about how you build your quote unquote roadmap, right?
And how do you go about deciding what is the next most important thing to build or to address?
Our product is very different from your average SaaS application. We will or are already becoming a research heavy company. I told you that the first prototype was built on top of third party services. We're now in the process of fine tuning our own machine learning models to do the same pipeline end to end.
The next iteration of that would be to actually build our own end-to-end model that we will definitely base on some of the existing research papers that do end-to-end speech-to-speech translation. But all of that is a very active research area. In a way, I know which step we're going to take in terms of increasing the risk and complexity of the machine learning pipeline step by step.
But it would be very hard for me to predict how long it would take us to prove each step of the way. And regarding the APIs or how our customers are going to connect to the core piece, which is the speech-to-speech translation, I feel like that part will likely stay relatively stable. And most of the efforts are going to go into just building the best in class speech translator.
So how do you go about or how are you going to go about building your team? What do you look for in those people to indicate that they're the winning horses to join you?
This is the third venture-backed company that I'm now building, so I feel like I have plenty of battle scars and tiring mistakes from my last two ventures. My motto for building an early startup team is lean and mean. And what I mean by lean and mean is hiring people that punch above their weight. I do not care about pedigree. I mostly care about people being honest
on board with the mission and willing to work hard and having a very tight knit of engineers that come to the office every day and have as high information sharing and collaborative environment as possible. And so if you ask me in a few words, what do I look for in an early hire? It is excitement.
It is willingness to work hard and definitely low ego because it just makes the collaboration a whole lot easier. And so that will be like a short answer.
This message is sponsored by SnapTrade. Link end-user brokerage accounts and build world-class investing experiences with SnapTrade's unified brokerage API. With over $12 billion in connected assets and over 300,000 connected accounts, SnapTrade's API quality and developer experience are second to none. SnapTrade is SOC 2 certified and uses industry-leading security practices.
Developers can use the company's official client SDKs to build investing experiences in minutes without the limitations of traditional aggregators. Get started for free today by visiting snaptrade.com slash codestory. This message is sponsored by QA Wolf. If slow QA processes bottleneck your software engineering team and you're releasing slower because of it, you need a solution. You need QA Wolf.
QA Wolf gets engineering teams to 80% automated end-to-end test coverage and helps them ship five times faster by reducing QA cycles from hours to minutes. With over 100 five-star reviews on G2 and customer testimonials from SalesLoft, Drada, Autotrader, and many more, you're in good hands. Ready to ship faster with fewer bugs?
Join the Wolfpack at QAwolf.com to see if they can help you squash the QA bottleneck. Given all of your experience and you were doing machine learning before it's cool, right? So you've been around this space for a while. I'm curious about how you approach scalability. And I heard the tools you were using and things of that nature, but was this built to scale from day one?
Or are there interesting areas where you're fighting it as you grow?
I feel like when it comes to scaling machine learning models or architecture, We're still very much in the early days. I feel like most companies are still writing their own infra once they get to the stage where they need to scale either their training infrastructure or scale their inference infrastructure.
There are tools built by big tech to make that simpler, but typically you run into limitations relatively quickly. And so, so far... I see most companies having to roll some version of the infra themselves just to fit their very specific use case. We plan to try to rely on kind of the cloud providers as long as possible. I don't want to build my own GPU cluster.
I think that's a waste of time for most machine learning companies. So unless you're building an LLM that costs you 10 millions or more to train, I think you're very, you're much better off relying on someone else's infra and not take on that DevOps responsibility yourself. So we're definitely going to rely on the GPUs that we get from the cloud providers in the early days.
And yeah, you know, you asked, have I seen the challenges? Yes, I've seen the challenges, but I feel like two years in the machine learning infraspace is an eternity. And so there will definitely be, you know, new frameworks that we'll have to use or will be new platforms that are probably going to be better pricing than they were a few years ago.
Yeah, rely on your past experience, but approach every problem with a fresh mind and willing to be surprised by something new, better, making more sense than what made most sense two years ago.
So as you step out on the balcony and you look across all that you've built thus far with Commonbase, what are you most proud of?
the optimizations that have gone to improve the latency of our very first product. Every customer call that we have, latency is always a question that comes up and it's always one of the main parts of the conversation. If you want to have real-time translation, it's always a question of, right, like, what is real-time? Is 2,000 milliseconds latency real-time? Is 1,000 milliseconds real-time?
Is it 300? When we initially got started, we had a very, let's say, flaky infra in terms of how we scale different streams, how we patch things together. I think I'm proudest of just very sober engineering decisions that led to much more stable infrastructure, that led to quite a lot of improved latency just from
optimizing using kind of 80 20 analysis and so trying to reduce networking time trying to reduce the the total number of api hops that we need to do between the api providers and our servers i think i've applied 80 20 very well and gotten a lot of bang for and not a not too much of engineering effort that's probably my
That's probably what I'm proudest of, not over-engineering the early version of the product.
Okay, let's flip the script a little bit. Tell me about a mistake you made and how you and your team responded to it.
We had this bug where the longer the call got, the longer the delay in the original speech and translated speech was. We could not figure out where that came from. It was a very strange bug. I initially thought there was a memory leak, but then we're using TypeScript as our stack, so I was a little bit surprised if it had been something that I would have had control over.
And I threw a lot of weird, different things against the wall to try to chase it down. I knew that there was a feature that I added very early on, which was saving the raw audio from both calls into an audio buffer and then saving the file ourselves at the end of the call. And I never thought that such a... Simple functionality could be the cause of this delay.
But then after a month of debugging and just like scratching my head, I just removed two lines of code to build the audio buffer up during the call. And all of a sudden the delay was gone. And I was like, Damn, that was literally like should have been the first candidate to like chase down.
I personally feel like those things are much easier to find if you describe your problem and bug to an engineer that has nothing to do with the code. So basically rubber ducking the bug with another engineer. That's probably one of the costliest mistakes in terms of customer questions and dissatisfaction in terms of not amazing demos to date.
And that could have been totally avoided had I just approached the bug with someone with a fresh look at the problem. So yeah, collaborate more. That's maybe the ethos of the story.
So this will be fun. It's early days. You are building a product that you're excited to be known for now after the pivot. Tell me about what the future looks like for the product and for your team.
So right now, I am heads down heavy building mode. So we are trying to get our own model out of the door and replace the existing setup while at the same time trying to improve our own infrastructure further. And so now we're on a stage where in addition to building on a daily basis, we are also trying to hire and expand the team here in New York.
Our office is located in Soho and we have an in-office culture. And so right now I'm spending quite a bit of my time on interviewing early hires. We are hopefully about to close our technical lead position and I'm interviewing a few machine learning researchers to start work on our own models.
It's really a combination of building and then hiring to just have a best in class product that would basically, let's say, never fail to wow the customer away once we get to that stage of the company.
And so I think we're going to be heads down building for the next three to four months and then likely raise the next round in order to start getting even more serious about the research and the training our own models part. Anyone building something early stage, I strongly believe that even the early versions of your product could wow the customer.
And if people ask me, why did I go into machine learning when I was able to do research in any part of computer science? I personally always felt good. The machine learning was the one part of computer science where if you showed someone what state of the art was able to do, an average person that didn't know how it worked, they would always feel like it's magical.
I did research on some like early image generation algorithms using gallons. And any time I would show what I was able to generate back in like 2017 using a neural network, people were blown away. I feel like that magic that machine learning has, it's still there. And that's the reason why I love that space so much. That's the reason why I've stayed in that space so much.
That's the same principle that I applied to the products that I built. I really want them to blow people away because that's the only way how to stay apart from the competition and in the sea of boring SaaS apps.
Okay, let's switch to you. Who influences the way that you work? Name a person or many persons or something you look up to and why.
The first person that comes to my mind is 100% my mom. My mom has always been one of the biggest inspirations in my life. I always look up to her because I just feel like she has always done a great job of being very high EQ and always understanding what makes people tick. But also at the same time, just being a very kind and hardworking person. That's definitely my idol number one.
I think like on a second spot, I would probably just like generally name like a few very close friends of mine that are all in similar spaces like I am building tech companies. And I feel like I have this group of four, five friends that I can bounce any problem off in my life, be it a personal problem, be that a relationship problem, be that a health problem, be that a work problem.
They will always take the time to help me out and vice versa. I feel like those people have been with me for years, and I'm sure that we're going to be there for each other for many years to come. And so I would say that small group of friends is definitely my inspiration and support rock number two in my life.
Okay, so last question. So you're getting on a plane and you're sitting next to a young entrepreneur who's built the next big thing. They're jazzed about it. They can't wait to show it off to the world and can't wait to show it off to you right there on the plane. What advice do you give that person having gone down this road a bit?
The same thing that I mentioned before. I want to see why they care about the problem. And I want to see how either the product blowing me away or their enthusiasm about the problem and how they're solving it blowing me away. We live in an extremely noisy world. And so every entrepreneur out there needs to somehow stand out from the sea of mediocrity.
And so I think really the two ways how to do that is either build something amazing or Or just be so excited about what you are about to build to make people excited to join your journey. Be that as a fan, be that as a customer, be that as an employee, be that as an investor. But basically, just blow people away with the first interaction first.
And make the world come along with your vision and mission.
That's fantastic advice. I certainly agree with that. Well, Icky, thank you for being on the show today. And thank you for telling the creation story of Common Base.
Yeah, thank you so much for that, Noah.
And this concludes another chapter of Code Story. Code Story is hosted and produced by Noah Laphart. Be sure to subscribe on Apple Podcasts, Spotify, or the podcasting app of your choice. And when you get a chance, leave us a review. Both things help us out tremendously. And thanks again for listening.