yes, the below line:
go func() { <-timeout }() // prevent leak
could also work, but it creates a separate go-routine which eventually will be finished depends on the timeout variable.
When the timeout variable was high then go-routine would exist during this timeout, which is not good for a performance. It is better to close a timer when we know that it is not longer required.

“`
timeout := time.After(time.Second)
select {
  case <-timeout:
// do something after 1 second.
  case <-ctx.Done():
go func() { <-timeout }() // prevent leak
// do something when context is finished.
  }
“`

You can find a working PySpark demo at: https://github.com/arangodb/arangodb-spark-datasource/tree/main/demo#pythonpyspark-demo

Hi Fabio,

If I understand correctly, this approach is using the DistillBERT model in python to calculate embeddings for documents which are then stored in ArangoDB.

Yes that’s correct!

I have seen elsewhere the use of ArangoSearch, which I think did tokenization and embedding directly in the database. Do I understand the difference between these approaches correctly?

Yes, ArangoSearch allows you to perform tokenization and full-text search directly in the database. At this point, word embeddings aren’t directly supported, which is what this tutorial lets you do. ArangoSearch does support vector space models such as BM-25 and TF-IDF for scoring search results. Please see here if you want to learn more about them.

The query uses the expression below to calculate the dot-product of the query embedding to document embedding. This implies a slower single-thread approach, though if ArangoDB is calculating this value for multiple documents concurrently under the hood it would still get the benefit of multi-core processors. Any thoughts/comments on performance?

Great question! The answer is that it depends. If you’re querying a single server, it will use a sequential scan (so a single thread). If you’re querying a collection on a cluster, and the collection is sharded across different servers, then there will be concurrency at a database server level, but within those server processes it will also be scanned sequentially.

hi Rishav,

Thanks for your comment.
We’ve chosen to offer a Free-to-try deployment that has enough resources to let you really try out all of the features of ArangoDB and Oasis.
That is free for 14 days.

The Developer deployments are aimed at individual developers.
If you need a fully free option, you can run the ArangoDB database in a docker container on your laptop or any small VPS.

Ewout

All other DBaaS providers have a free tier and frankly as developer somewhat interested in trying Arango cloud, I don’t see any reason to try it out if I can’t tinker around with it for my pet projects.

This developer account is welcome, but is not what we really want.

Hi! Thanks for the response, if you want to check out this following post it is more up to date and can help answer your question https://www.arangodb.com/docs/stable/foxx-reference-modules-graph-ql.html.