Great to see more love for using Lucene on Lambda via GraalVM native binaries! Explored that same idea few years back for my personal blog [1]. It worked pretty well, but I moved eventually away from Lambda as I wanted to close down my personal AWS account. One thing I always meant to explore was embedding the (in my case small) Lucene index directly into the binary image using Lucene's RAMDirectory via Quarkus' bytecode recording machinery, but I never got to it.
Fascinating. In a modern app I find that search quickly can become one of the most expensive aspects. It’s not trivial to implement on your own and with the growing effort on hybrid style search, the costs continue to grow. I love work like this.
Surprisingly, 3GB is a real practical RAM limit for aws lambdas in 2025: you can only have more than that if you submit a support ticket. But it's not really mentioned anywhere in the docs.
The default Lambda quota for all accounts is 10240 MB. I've never seen it below that (in recent memory, at least), even on fresh accounts not connected to a big org.
I know I routinely use 10gb of RAM for my account that's never talked to support for the related CPU allocation.
Serverless gets a lot of hate for its name, but I have had so much success with it myself that I have moved on to its successor and have started developing for computerless architectures.
It is the next level of abstraction for sure. Cloudflare seems to be making the biggest strides in also abstracting regions and the geolocation of the data, but I’m not super familiar with their offerings.
Great to see more love for using Lucene on Lambda via GraalVM native binaries! Explored that same idea few years back for my personal blog [1]. It worked pretty well, but I moved eventually away from Lambda as I wanted to close down my personal AWS account. One thing I always meant to explore was embedding the (in my case small) Lucene index directly into the binary image using Lucene's RAMDirectory via Quarkus' bytecode recording machinery, but I never got to it.
[1] https://www.morling.dev/blog/how-i-built-a-serverless-search...
This is great!
I have been using Quickwit as a low cost search engine on Lambda. It works very well for my relatively small and infrequently updated dataset.
Unfortunately Quickwit devs have decided to not support the Lambda deployment mode going forward so eventually I'll need another option.
Fascinating. In a modern app I find that search quickly can become one of the most expensive aspects. It’s not trivial to implement on your own and with the growing effort on hybrid style search, the costs continue to grow. I love work like this.
Maybe a stupid question: why can't the indexer write directly to EFS, why does it write to s3 and lambda downloads it to EFS?
"128MB default with up to 3008MB max. You can submit a support ticket to get 10GB RAM, but I was too lazy to argue with AWS support."
Was this written before wide availability of 10g memory lambdas?
Surprisingly, 3GB is a real practical RAM limit for aws lambdas in 2025: you can only have more than that if you submit a support ticket. But it's not really mentioned anywhere in the docs.
https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-...
The default Lambda quota for all accounts is 10240 MB. I've never seen it below that (in recent memory, at least), even on fresh accounts not connected to a big org.
I know I routinely use 10gb of RAM for my account that's never talked to support for the related CPU allocation.
https://aws.amazon.com/about-aws/whats-new/2020/12/aws-lambd...
Is what you're talking about a new thing? Or respectfully, are you just wrong?
They probably have never requested service limit increase to unlock all of AWS.
If they had, they would know that it involves many weeks of arguing with support, of course
Justifications upon justifications, man, so glad I no longer run infra.
Serverless gets a lot of hate for its name, but I have had so much success with it myself that I have moved on to its successor and have started developing for computerless architectures.
It is the next level of abstraction for sure. Cloudflare seems to be making the biggest strides in also abstracting regions and the geolocation of the data, but I’m not super familiar with their offerings.