Shay Banon
Appearances
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And when it gets to a point where Amazon Elasticsearch exists for enough years, it becomes a fact whether they end up changing their name or not.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Exactly.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yes, and especially when it wasn't even it's Amazon's Elasticsearch version or something like that. It's done in partnership with Elastic, right? Like that was the additional layer of confusion that kind of like made it really big. And that was really confusing to users because it's like, because if I naively look at it from the outside and I go Amazon Elasticsearch, you go like, oh, amazing.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Elastic managed to do partnership with Amazon and they take Elasticsearch and provide it as a service and I'll go and use that. And, you know, we all think that we live in hacker news on Twitter or something like that, but most users don't. They just go and say, oh, Amazon's Elasticsearch. I'll just go and use that.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And they don't understand the nuances or the fact that it's not us or something along those lines. And, you know, nothing bad with Amazon as well. You know, they're a great partner of us now and we work well and actually think that Amazon matured a lot in how they run their cloud versus the early days. But we just felt like we had to make a difference and that's where we landed.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah, we changed it actually to two licenses that you could choose from. The SSPL one that MongoDB changed to. So MongoDB did that change as well. Or a license that we wrote that was more in the Apache type. You know, there's kind of like two families of licenses. We call it the Elastic License V2. It's actually a very, very simple license.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And we allow users to choose between them because some were concerned about the SSPL, someone concerned about the Elastic License V2. We kept all of our code open. We kept all of our free stuff free. We just didn't call it open source. Nothing changed except for the license, if that makes sense.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
First of all, I was very sad and it was painful. And, you know, as someone that loves open source and believes in it, that was a sad thing not to be able to call it open source. And I agree, by the way. I think if I understand things correctly, we could have called ourselves open source because there's nothing illegal by calling ourselves open source.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
But I believe in the social contract of open source and OSI. And we stood by it and we changed our name and we, you know, search and replace all open source with free and open and other stuff. And that was not great. Our usage, interestingly, you know, we live in our own like bubble of interest and, you know, and things like that. Our usage didn't change that much to be completely honest.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
So the usage of Elasticsearch continued to grow. People just like downloaded the Elasticsearch and used it and ran it and it was free and using it for search and all the other use cases that people use it for. But we were still not called open source. And I think open source matters a lot. And we always hope that we will be able to get back to open source.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
When it comes to our brand, there's many people that I love in the open source community that were really upset with this change. And that hurt. Not the fact that they were upset with the change. That hurt because I made them feel like that, if that makes sense.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And that was sad for me. And there were certain companies that say that they can only use open source that stopped using us. But generally, we continue to grow as a company because the truth is that most users actually use open source software, but they don't engage with the community or contribute to...
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
issues or something on those lines, the portion that do I adore, and we do, we did lose some of them, sadly, because of that. And that's one of the reasons why we're back to being open source, because we want to, you know, we want to bring them back.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
It wasn't a big detriment. I actually think that because now Amazon Elasticsearch got renamed to OpenSearch and Amazon OpenSearch, then it's much easier for us and for our users to know what do they use and then evaluate as one versus the other. You know what I mean? It's Elasticsearch versus OpenSearch. you know, figure this out.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Like we had with Solar, like we have with Splunk, like we have, you know, with other products, it doesn't matter. And I think that's totally fair. I do think also that open source is also great for when new markets get created. And you have the hush puppies users, you know, the leaders, the new ones that start to experiment with types of software.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And I've been thinking about open source, getting back to open source for more than a year now, personally. But certainly something that helped make that decision is the rise of vector databases in the Gen AI space. Because I thought that being able to say Elasticsearch and open source vector database is an important thing to say.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
So that definitely helped in the decision to get back to open source.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
It's not as strong. There's, there's, you know, there's like so much in open source, you know, like there, there's so much beauty and romantic aspects and history and in just in these two words that they encapsulate so much, you know? And it's like, and I love it. It's like, that's that. Yeah. Yeah.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I think it's fair that there's more companies changing the license today because they're really worried about cloud vendors. And, you know, it's funny to me that there's not more of, hey, this is real pain that you're feeling. Startups, you know, I developed Elasticsearch in my free time in my living room. You know what I mean? It's like... And somehow, I don't know, maybe it's human nature.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I find it funny that people take the side of Amazon in this case. I found it hilarious. But I think there's real fear. And we need to acknowledge the fear. And if the outcome that we want is more open source in the world, why do these companies change it? It's not because they IPO'd.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I'm talking to tons of startups that go and say, I'll never open source my code that would have open sourced the code because they say, we don't want the cloud provider to come in and take all the stuff that we built and pulled something around it. I think there's a few companies that says we need to bring trademarks back and enforcement and things on those lines. All of these are good discussions.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Those are the discussions that we need to have. What I can say, at least with our change, and go back to what we spoke about in the beginning, I think that enough time has passed from the hectic days of early cloud that AGPL is potentially good enough protection. AGPL is the license that we chose to be open source, and it's potentially a good enough protection.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
So when I talk to companies today, I say, go open source, build things in the open, choose AGPL, and it's probably good enough protection. Because we're choosing a GP, you know what I mean? Because we're showing that that's the case. So hopefully that changes their trajectory of choosing an open source license and which one ends up being chosen.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah, I think it's a good question. MongoDB that created the SSPL license. So MongoDB changed their license from AGPL to SSPL, being concerned that cloud providers will take AGPL software, will test AGPL basically, right? So they change it to SSPL.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
So even from then, I think enough time has passed that I think when you put AGPL out there, it's a good sign for cloud providers that this is the contract that you expect to have with them. I think SSPL tries to solve it. MongoDB went through a whole process, and you can read the mailing list discussions and things like that around trying to get SSPL approved by OSI. It didn't happen.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I'm not pointing fingers at one side or another. There's a lot of emotions and the devil is in the details and what have you. I think there's a good question. I think there's a balance that OSI tries to put around what is open source and how do you define open source, which is very important to maintain. That social contract is important. There's a lot...
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
writing on the term of open source that we should cherish it and treat it very dreadfully, if that makes sense. And that's like rushing to adding like five licenses to it and it's fine. And then it becomes, you know, if we do it, it becomes meaningless. Right. So we tried to figure out if there's an option with OSI to maybe revisit the SSPL discussion.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
They have a new leader and I think they're open for it. And they're trying to figure out what is the open source, how does open source live also in a world not only of cloud providers, but in a world of models and open weights versus open models and open data and things like that. And how does open source play in it? They have a lot on their hands.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I think they want and they really want to try to figure this out. But for the same reasons why we felt like we had to change the license because we don't have time, I actually think that we're in a position where I'm not worried that someone takes Elasticsearch and provides it as a service. So I can conceptually take the very, very minuscule risk of just going with AGPL.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And potentially with that, we're actually creating a fact. And it's like, that's it. AGPL is fine. That's how it works. And you don't have to go through the process of approving a different license.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
So I think that sometimes we make the mistake of judging open source based on, you know, it's like we always like the songs when we were teenagers and they're stuck with us and it's like, it will never be better. So it's like some of us, remember open source licenses and usage like it was in 2004 and five or three and seven.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And it's like, how many hours did I spend around the differences between GPL and Apache? That was within open source, by the way. And it's like, oh, you can't choose one versus another. I think developers today, they pick an open source license and they're fine. They're fine. You know what I mean? It's like they care much less about the fidelity of difference between Apache, for example, and GPL.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I actually also think that big companies care much less about, like enterprises. We used to think like, we used to say Apache is much easier for enterprises to adopt, but But look at MongoDB. It's all over the place in large enterprises and it's AGPL. So it's like, obviously that's not hurting adoption or was AGPL. So it's not hurting a job adoption.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And I think a similar thing might happen in AI and that might be the missed opportunity. We see open models, we see open this, we're saying open source AI, open source models when they're not strictly matched the definition of open source.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I don't know about, it doesn't matter about OSI or not. I just don't want the term open source to get diluted.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And I think that's the responsibility of OSI and us, by the way, to help, to try to figure out, because we obviously play a big role in Gen AI as well and building models and things like that, to figure out how do we build a system that allows for it? Because you just don't want it to be diluted to become a term that means everything.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
It's open source.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
It's like a princess bride. It's only slightly dead. Yeah, exactly.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Pretty much, yeah.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I think there's a lot of urgency in being able to properly define it. So we as a society own the terms open source and it defines what we do. I will say this is not like an us versus OSI. That's not at least how I think about it. I think OSI is doing a lot of amazing work and it's our responsibility to help OSI to figure this out. Whether it's monetarily...
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
where we can contribute to OSI so they have more people and more money to help figure this out, or by working with them to try to figure this out together. I do think in such a new space that moves so fast, like Gen AI, like models, open source has been hijacked, the term, from OSI.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And I think it's important to try to figure out how do we consolidate that fact that is happening as we speak.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I think it's more than the megaphone that Meta has, right? Because Meta, for example, released quite a few open source projects over the years.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And if they would have called one of them open source when it wasn't OSI approved, I'm pretty sure that we could have resisted it. I think the other factor that is at play is that's a model that they're putting, under a very permissive license that cost hundreds of millions of dollars to build.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
That's pretty crazy. In a very short amount of time, I didn't do the math of how many money did we spend Elastic as a company and personally on just building Elasticsearch. But in a very short amount of time, a lot of money was spent to build our product. And this is pretty remarkable. to be able to put it under such a permissive license.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
So that's another reason why people go and say, okay, we want to use it.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah, and I think the interesting part is that, for example, they made the license more permissive just recently, right, if I remember correctly.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And I think this is the important work that we need to do because open source within models might have a lot of fidelity. There's the weights that might be open or open source. There's the retraining opportunity that you might be allowed to do or not. There's the data sources that you use to train the model.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
They don't have to be open source, but are you opening which data sources, like just listing them so you can go and certify them or not? So there's like... A lot of areas in these places that if we have a standard way to think about them like we do with other open source licenses, I think it will be good for the world.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I think there's a meta question around if we're going to have really like... It's a question also of how many models are we going to have, especially like LLMs, like large language models that are really expensive to build. If we're going to have five... then maybe it doesn't matter. And people go and say, oh, we know the Lama one. We know the challenge that way.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah, they call it open source, but I know, you know, we know the restrictions. If there's 20 or 50 or more specialized models and things like that, that's when I think we really urgently need a common way to understand what is our freedoms and what do we get when we use a specific model. It's really important to understand it in a standardized way, if that makes sense.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
It can do what we do. It can just AGPL it. And then it's like, if you end up using this LLM, then you need to open source the rest of your infrastructure as a source.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah. Ah.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Obviously, any change that you do is scary. And it's like, the easiest thing to do is not to fight inertia, right? And I won't lie, internally, people were saying, why do we need to change it? People got used to it. Why do we need to go and change to open source? So these decisions are not straightforward. But we really felt like we are an open source company.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
We walk and behave like open source and this is who we are. And this goes all the way back to our value system as we were founded. And yeah, we announced it and it was an amazing experience. I was scared. I was obviously happy about being able to announce it. I was scared that people would twist it because you know how the whole social discussions might happen.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Oh, Elastic is in bad shape or they're choosing open source or something on those lines. And And sometimes people really liked it, but it was just pure joy. The vast majority was just like, this is amazing. Thank you for doing it. Welcome back to open source. Prominent open source people were engaging and appreciative of doing it like Adam and others. So it was great.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
To be honest, I'm very happy. I don't know if we'll describe rug pull, but I'll go with it.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Obviously, there are still people that try to find the bug in something good, but I think there always are. I try to focus on the fact that this is just a net positive progress for Elastic in bringing back open source. And it's hopefully an even bigger net positive for open source world because hopefully it will convince more companies to do open source versus less.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I think there's a few reasons. The first one is we never stopped behaving like open source, right? All our code is still on GitHub. All of our issues are on GitHub issues, pull requests, reviews, people send us pull requests. You know, it's like never stopped.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And if you go back to ask yourself why you changed the license, we changed the license because of the trademark, but the trademark is no longer an issue because Amazon changed the license to open source.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
So really, the only question is why not in that case, which because if we're not afraid of Amazon calling it back, Elasticsearch, which we are not because we work well together now and they went with OpenSearch and just totally fine. Then I think the question is sometimes like, why not? Sometimes the easy one is why not?
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I also totally believe in the value of open source. As I mentioned, there's so much encapsulated in these two words and so much of it encapsulates who Elastic is as a company with our contribution to open source. Our many open source projects from Logstash and Kibana. Kibana got forked into Grafana and created a whole other company. It's like...
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
There's so much that roots itself back to Elasticsearch. We invest so much in Apache Lucene, which is an amazing story in the open source world and an incredible one. There was always a hole left in our hearts when we took away open source and we just felt that we wanted to fill it, if that makes sense. I don't know. Yeah.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah. Whether you like it or not, it is rug pull.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
So I'm very excited. And I think as someone that has been in the open source world for, gosh, 28 years now or 25 years or something like that, I love open source. And I think having less open source in the world is bad and having more open source in the world is good.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I think so.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I still would like to take back the years that we were not open source. Yeah. It worked out, but not in the way that I would have loved it to work. You know what I mean? Right.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah. I mean, we had a great story around how to get people on our cloud. We had a great story around like commercial features on prem, like the whole business model that around open source that we, I think we were a big part in building as a company and as a community, you know, with other open source projects like this work, like you don't want that headache.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Under Apache 2, yeah. Under Apache 2, they're allowed to do it. And obviously, as long as we have Apache 2 license, I'm not going to go to Amazon and say it's illegal. Right, because it wasn't.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yes. So some people say it's great, but it doesn't align with the norms of open source or something along those lines, which is fine. It's still legal to do it. Our case specifically was just around the trademark.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yes, yes. My son is a huge Kendrick Lamar fan. And as a result of it, I became a Kendrick Lamar fan. I mean, I introduced him to him, but I didn't know him as well as I do now. Gotcha.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
So that was like running through my head when I wrote it. Yes.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
No, definitely. I sat down and wrote that blog post I care about. You know, it's like, you know, maybe some people made fun on Twitter on this, like, why is he doing these references or whatever? But, you know, it's like one of the things that I love about open source is that we're quirky. And we're like adding some fun to the stuff that we're doing. And it gets interesting.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And I think... I hope that what we do at Elastic will help change some trajectories that we've seen happen with other companies and cause for having more open source in the world versus less.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And it's like, that's the thing that I love doing in IRC or Slack or GitHub or doesn't matter. SourceForge, you know, it's like all of these things. We had fun building software. And, you know, it's like that's the thing that I cherish a lot. And this is part of it. At least that's how I read it.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Totally. That's already interesting. And that's fun. That's why we're having like fun discussions. That's what we should be having. Not like heavy, depressing, accusational discussions. You know what I mean? It's like, that's the fun that we need to get back to having.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah, I think details matter, as you said, and for people that are interested in it, it matters. I think the end result of having more open source in the world, it's much easier to just all agree on. And I think if we're all like,
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
the arc of it is where we're heading i think it's it's goodness right and to be honest like i actually think that like if you build enterprise software that's my recommendation you can pick agpl if you want to go open source and not worry about the cloud stuff as you we discussed i'm more worried about the large language models and ai area which we need to figure out and
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
That's an area that we also need to invest in. And hopefully this podcast also helps push it in the right direction, if that makes sense.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I think that people are going to see a significant value in their search applications by embracing semantic search and vector type use case. That's the opportunity. I always like to think positively around the opportunity. And I think the opportunity is huge. If I loved search as a way to liberate your data before, this is like 100x better. And when we...
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Obviously, we at Elastic and we're a huge contributor to Apache Lucene, which we use. we added HSNW and vector capabilities to these systems years ago, right? Before this whole gen AI happened. But once it happened, obviously the weight of being a great vector databases has increased significantly because the improvements that you have in search capabilities is significant.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And we're saying internally at Elastic that we want to make Apache, Lucene and Elasticsearch the best vector database in the world. And then on top of it, the best hybrid search, because it's like it's not only vector search, right? It's like there's a lot of layers on top of it to make a great search system that we've been building for years.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And yeah, I mean, like any new type of capability, I think you end up having like new competition or other new vendors or products, open source or not, as you say. I think Elasticsearch is a... Great vector database today. In Apache, we contributed so much work into Apache Lucene.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
We have some of our best developers working just on Apache Lucene on making it better and then integrating it into Elasticsearch. By the way, by making Apache Lucene better, we're making a lot of other systems better. So it's like, our focus is to put the right technology at the right place and then win based on value.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And yeah, and so far, by the way, that's the fastest growing market in the world, in our business. So that's definitely the fastest part for us in terms of usage and business. And people are seeing the value. I do think one of the things that I'm trying, working really hard to figure out is one of the things I think Elasticsearch did really well is that it made search approachable.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
You could just dump a JSON document into it and start to search it, and you didn't have to understand vector space and ranking and BM25 and inverted indices and FSTs and all of these things. And then we progressively disclosed complexity if you wanted to or you wanted to tweak things, but it just worked.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And we've invested over the last year a lot of effort, not only in making us a great vector database and great in hybrid search and things like that, but also making it as easy as it was to index. We literally took the curl requests that I had in the first blog post when I released Elasticsearch around indexing tweets, and we're literally trying to make it this simple to do semantic text.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And I think if we go back to hundreds of thousands, if not millions of developers that know Elasticsearch and know Elasticsearch API, that's a huge opportunity because they don't have to become experts in vector databases and product quantization and HSNW and dimensions and embeddings models and things like that.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I was always saying that that's probably the hardest decision that I've ever made at Elastic. So that was a really tough decision. And like any tough decisions, you don't come to it lightly. Like you understand the implications, you understand the impact. Also, just on a personal level, it's like the open source has been a big part of my life.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
We can really simplify that layer to them and they can use Elasticsearch the way that they love.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And to me, like, that's a huge opportunity because if we make every single Elasticsearch developer out there, like someone that uses Elasticsearch, just use semantic search, even without, almost without noticing, because it's so simple, like that would be amazing for them because it's significantly better. And for us as a, as a, as a product in Elasticsearch and as a company.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I think it's here. So we, I don't know, I'll get geeky for saying, but we have like a, when you define something in Elasticsearch, there's mappings for it, which is basically the schema. And then there's like text mapping, which basically does keyword-based like analyzer, BM25 type matching. If you do semantic underscore text and don't change anything else, it just becomes semantic.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And we pick the built-in embedding models for you and we run it and we do everything out of the box. But every single layer is open. So you can change the embeddings model that you use, you can change the way that the query is being generated, you can change the You can go all the way down to just providing the vectors yourself if you want to. But like the simplicity at the surface area is there.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And we worked really hard for the last year. That required significant investment for us. And by the way, all of this is free. So all of this is in the open source now. I can say open source and just go and start to use it. And if you know Elasticsearch and you know how to index data, let me tell you a secret. You know how to do semantic search now. Wow. So that's that's exciting.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah, happy to. Thanks for thanks for hosting me.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I was trying to calculate back like how many hours I worked on open source while I had another job. And I think I was... Since 2004 or 5, I've been basically working like 30 to 40 hours a week just on open source, whether I had a job or not. When I didn't have a job, I would work more. And that's a lot. And I do it because I love it. I do it because I cherish it. I love the community aspect.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Oh, yeah, yeah, yeah, yeah, yeah. Yeah.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I mean, I can say, sadly, I know more about trademarks than I want to. There you go. School us. So that's totally a legal usage of trademarks.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
That's totally illegal. If you're allowed to say, I'm a shop that service Toyota cars, you're not allowed to say, I'm a Toyota service shop. That's the difference, right? And as long as the name is Superbase, they're totally fine to say, it's an open source version of Firebase. That doesn't create confusion in the market between Superbase and Firebase. And that's totally allowed. It's fine.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I actually encourage that type of marketing for any open source projects that there is. Go for it. That's cool.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I love the engagement. I love building software that is accessible by so many users. Yeah, yeah, I love it. So that was a really tough decision. It also, it took time to get to it, you know, because you always hope, kind of like hope that somehow you won't have to do it.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
When you get caught in that, that's hard. And sadly, it's easy to bucket things. I don't blame. I tried to explain it in our not okay blog post and be very blunt and explicit about it. We actually made a mistake of... Our first blog post around the license change was bad because we were like, we just don't want to make a lot of noise about it or something like that.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And we made a mistake of not being explicit around why we're changing it. And the day afterward, I wrote this not okay blog post. I think the social streams, just like simple messages and bucketing, and then it kind of took a life of its own. And it's very, very hard to change once it's happened. I don't blame any individual. You know what I mean?
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
It's like I get caught by it every once in a while.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
But that's okay. I think that's life and you have to figure out how to work through it, if that makes sense.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And hopefully, I don't know, Amazon will change the name of Amazon Elasticsearch or somehow it managed to work some sort of like an agreement with Amazon or something on those lines. But...
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I think so. I think it would have helped. Yeah. But I also think that getting to a point where you sue someone is never helped. You know what I mean? And I think truly, I think Amazon, where they landed in a gold mine called Cloud, and they were like running very, very fast for the first few years. And you just took Elasticsearch, provided a service and called it Amazon Elasticsearch.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And then I think it was very hard for them to decide to change it. And they wanted to get, I don't know, it's hard for me to explain why they refused to do it. But you can see that they learned and they evolved and they didn't do that level of naming afterwards for other services that they took and provided as a service. And to be honest, like now we work really well with them. It's funny.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
It's like... It was very hard for us to work with Amazon on the marketplace, for example, because when we would go to the marketplace and we would say, like, please promote, let's work together to promote our Elasticsearch hosting service. And they'll say, but there is already one. It's called Amazon Elasticsearch.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
So it's like even trying to work with them was hard because of the confusion, because the confusion was internal because Amazon is so big. So once the name thing got cleared up, it just unleashed also the opportunity to even start to work together properly as, as two companies.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
When it gets to a point where so many users just confuse, either because they think you work with Amazon, either because they think that Amazon Elasticsearch is Elasticsearch or your cloud service or something along those lines, that we felt like we had to just go and change the license so it will force a change of name and a clear distinction between the two products.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I think that would have been fine. I don't know. Maybe it's kind of like hard to know. Maybe we would have ended up doing something similar to what Grafana did and change the license from Apache 2 to AGPL. So still remain within open source, but try to be more restrictive around whether Amazon really wants to take the software and provide it as a service.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I'll get back to, I do think that there is an imbalance in building open source and hosting open source. between the large cloud vendors and open source companies. And if we want more open source in the world, I think we need to figure out how to fix that imbalance. And that's a valid concern that companies have. And sometimes the way that I think people...
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
talk about it, it's very, it creates tensions like, oh, all of these like big IPO companies, they IPO, they were successful and now they're like changing their license because they have shareholders or something along those lines. But to be honest, I was never worried about this. I was worried about the
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
five startups with two people that I'm talking every week that don't choose open source, you know? Or just like people that live there and go like, we'll never open source our software. We don't want to happen to us what happened to Elastic, you know?
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I hope so. Because being very empathetic to OSI, they have to deal with the AI stuff and that's tough. And, you know, we donated money to OSI and we're trying to help, but that stuff, and I think it's more urgent to figure out than, you know, approving SSPL or something along those lines. So I do see a path where AGPL is good enough as a solution, if that makes sense.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
I think they're very interesting. They're very similar to like Elastic License V2, and we're talking to all of them. It's still the... OSI will never approve these licenses, right? Because of the restrictions. Yeah, exactly. I think what MongoDB tried to do with SSPL is to actually create a very valid license that had a chance to get approved by OSI.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And I think that's the path that I would want to take. Not like expand open source to allow for like 50 things get into it, But more around, let's figure out how do we still keep the high value open source term definition and apply it to AI, apply it to enterprise software.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
That was not there. I wish that we didn't have to do it. I wish that there would have been another way to solve it. And I know a lot of people have a lot of ideas on Twitter on how this could have been solved. Oh, yes. But I can tell you that as someone that was one of the hardest decisions, if not the hardest decision that I've ever made, we tried a lot of them.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah. Yeah, I mean, cloning closed source and trying to replicate and reimplement. Well, at least until we can go and chat GPT prompt something that says, create me an open source version of X. Yeah, exactly. But yeah, I think that that's true.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
By the way, the same is true, you know, it's like when people say rock pool, and by the way, I totally understand where it's coming from because this is like, this is how it feels. And you never argue with what someone feels. You know what I mean? It's like, it's not fair because it's like, that's the first impact reaction and that's how they feel. Then it's your responsibility to figure this out.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
But I think one of the beautiful things about open source is the ability to fork a project. And the ability to change it and the ability for it to grow and become something else. And when you change the license, then someone can go and fork the project and suddenly like a whole other project comes up and you remember, I don't know, what was it like? The best example from my old days was Jenkins.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
You remember? There was like Hudson and then Jenkins came along, got forked, and Jenkins became the thing and it worked. And we're seeing some examples of it today. And I think one of the points of reference on why when we changed the license, we still behaved as open source is that no fork got successful, including open search. Because we still behaved like open source.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
We didn't suddenly put everything behind closed source. We didn't suddenly make everything commercial. And I think most users were fine with it the same way that most users are fine with Lama as an open source model. You know what I mean? And I think that's why it never happened to us. But yeah, maybe for other companies it might happen and it will be amazing if we can solve it.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah. And I think Valky for Redis.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
That's another good example. But this is where I think most users saw how we behaved in the next few days and they were like, nothing changed. And they were less worried. Obviously, a lot of the people that are in the details and go back and have opinions about open source were hurt, justifyingly. I was hurt. But most users were like, it's fine. It's Elasticsearch. We know the team.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
We're talking to all the developers all the time. It's like nothing really changed.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And we felt like we had to go and do something about it.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yes.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah, yeah, yeah. Happy to.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yes. And until you understand the complexity of going through the legal process, if that makes sense. So I think there's two aspects to it. I think the first part is if you go back to open source and I'm relatively old school open source, a big part of building an open source software was registering a trademark and owning it, right? It's like Red Hat did it and others. And
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
The first thing that I did even before I released Elasticsearch was take a loan and register Elasticsearch trademark. And I don't know if you know, but there's many different types of trademark, but I actually registered two types of trademark, one for downloadable software and one for software as a service.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
So even back then, I wanted to make sure that when we build or someone builds Elastris as a service, it doesn't get confused. And at least for a while, people respected trademarks, you know, Apache and Red Hat. And that was like a wonderful protection because you could go and fork the project. You can do and do all of these things, but it's just different from that point on.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
It's a different project. I think cloud changed the game. And I think because Elasticsearch was so popular and Elasticsearch was one of the first services that Amazon decided to take and provide, I don't know. I'd like to think the best of people and afterwards try to figure out why not. I think they just took Elasticsearch and said, we'll call it Amazon Elasticsearch.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Maybe they didn't have experience with open source. They didn't have experience with this type of things. You know, naively, some product manager takes a product, provides it as a service, and that's the name. Right. And I think that you can see that they learn after that, right? Because when they took other open source projects, they called them differently.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
They call it ElastiCache and they call it Amazon Managed Service for Kafka versus Amazon Kafka and things along those lines, right? But Elasticsearch was one of the first ones and that name stuck. And that part, I don't know, for whatever reason, Amazon just refused to change the name, regardless of how much we try to talk to them and try to process through it. And you'll have to ask them why.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And basically it was kind of like the answer is I'll see you in court. And people think, oh, you're a big company and you have lawyers and you can pay for it and go through the process and whatever. But it's exhausting. You have to give testimonials and you have to do this. And it's like at least a few years of process. And, you know, it's like maybe it gets resolved.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And we ended up resolving the trademark infringement. By the way, thanks to the fork because it made it really easy. But it just takes a long time and it takes a lot of time. attention. And it just, it never felt right to me. I want to be focused on building great products, not, not like doing lawyery stuff, if that makes sense.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Definitely. I mean, like any good open source community member, I've spent a lot of time thinking about licenses since 2005 and LGPL and Apache and the difference between GPL and Apache, even within the open source licensing world.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Yeah, the fun stuff is to build products, to talk to users and figure out what their problem is and try to go and figure out how do you solve them or sometimes think slightly ahead of where they're heading and build it. Yeah, it's fun. That's where the fun is.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Totally, potentially. But I think the thing is that... Elasticsearch was under the Apache license. Amazon was totally within their rights to take Elasticsearch and provide it as a service. That was never the issue. I think that sometimes companies have a problem with strip mining open source by cloud providers.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
There's that saying, and it's not fair to take open source software and provide it as a service. And then it's like, there's a company like Elastic that invests so much and And other companies that don't invest as much in the open source end up ripping the benefits of it. But to be honest, it's legal. It's like if you have an Apache license, then what they do is what they do.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And that's totally fair. The part that bugged us is just the confusion. It's like that's the thing that was really weird for us.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And that's where the trademark comes in.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
They could have decided to. I think the world has changed. I do think that the early days of cloud were kind of like a bit hectic, if that makes sense. And I think norms have been established around which projects to use, which licenses are we going to use or not, what's the intent of the open source project, how to think about licenses. I wasn't worried about Amazon taking...
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Elasticsearch and breaking the license, if that makes sense. It was pretty obvious to us or to me specifically that Amazon would just decide to fork Elasticsearch, which was fine because a fork means a different name and then that name ends up slipping back into the Amazon service and then It's great. We compete. I love competing.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
Amazon had a cloud search product before Amazon Elasticsearch, and it was fun competing with it. It was based on Apache Solar. That's great. So yeah, if we can compete, I'd love to compete. It's just hard to compete with yourself. And it's hard to express how frustrating it is. You know what I mean? It's like you see in the forum, oh, Elasticsearch sucks. It doesn't run as well as it does.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
And then you go, oh, it's running on Amazon Elasticsearch. You know what I mean? It's like, it's not what we built. It's not how we run it. It's not the expertise that we have when it comes to running the service, even at that level. So that was difficult. And yeah, we try to figure out how to solve it.
The Changelog: Software Development, Open Source
Elasticsearch is open source, again (Interview)
It potentially could have been. I know that we tried a few years to try to resolve it outside of the court because nobody wants to go to court and get bogged down by it. When we were talking at least to our lawyers, it was going to be a multi-year effort.