My Open-Source Valentines 💕

These are a few of my favorite libs 🎶
recommendations
libraries
tools
Published

February 14, 2021

Open source is growing 📈🚀

And I’m guilty of being easily distracted by new shiny tools, libraries, projects… or whatever you wanna call them, each holding promises for greatness.

So eventually I end up 🌟starring way too many repos🌟 on Github…close to 1K+

With more awesome- lists being made, with more AI landscapes shared, trying to navigate among the stars can be overwhelming 😧

But among all those starts here are a few that made it into my favorites that I use daily, some I have learned about as recent as just last year. And some stable ones that I bet you already know.

Jupyter

I 💖 notebooks! And It is okay if you don’t

Jupyter is more than just notebooks. There is a whole jupyter ecosystem💞~ kernels for various languages, kernel-gateway, jupyterhub…

Recommended JupyterLab extensions: - jupyterlab_autoversion: a lifesaver for when you stupidly over-confidently delete stuff. - jupyterlab_code_formatter: Best code formatter for jupyterlab. - qgrid: If you work with a lot of table data. - jupyter-server-proxy: for running things through jupyterhub. - jupyterlab-lsp: brings jupyterlab one step closer to being a full IDE; code completion etc. but in my experience it can be bit resource extensive and slow.

Top magics: %debug,%history,%env,%timeit,%%bash … I want to get better at remembering/using more magics.

🌟 Community member highlight: Kevin Bates with my thanks for helping in my github issues

Streamlit

Yes, streamlit gets its own paragraph!

If you haven’t heard of streamlit🔥 what are you doing with your life?!😱 stop reading and go check it out

I learned about streamlit last year and fell in love. In a short period of time it has shown great growth while also building an awesome community.

Recommended streamlit components: - streamlit-drawable-canvas by Fanilo Andrianasolo - lol cause he beat me to it (very usefully)

There are some alternatives out there and I bet there will be even more in the future: - Gradio - Anvil - semi open-source, different logic to streamlit.

🌟 Offers simple hosting

Singularity Containers

I feel like Singularity is one of the unsung heroes for HPC. Think docker containers but more flexible…and dare I say easier to use. - no need to bind ports - you don’t need sudo to use it!

🌟 Community member highlight Vanessasaurus with my thanks for always helping in my github issues.

Note

I will do a post on using singularity containers for ML in March~ stay tuned!

When you have a need for speed

  • Numba: JIT compiler that translates a subset of Python and NumPy code into fast machine code.
  • Nvidia RAPIDS: CUDA accelerated operations FTW ⚡
  • Nvidia DALI: Speed-up your dataloaders. Learn more in my previous blogpost

Other Tools

Caddy

A super simple alternative to nginx. Because I’m a dummy when it comes to network stuff 🙃

  • Automatic HTTPS redirect + tls certificate management
  • Use as a reverse proxy, webserver
  • Easy to setup authentication. (using auth-portal for caddy v2. OR JWT token auth using loginsrv for caddy v1)

Node-red

This is an oldie but a goodie in my toolbelt.
Differently from other python-ian projects I mentioned this is for javascript. But you can also use python using external nodes. It is geared towards IOT applications but I usually use it to create simple event flows (using javascript and/or python) or dashboards(like a gpu monitor) etc.

If you like like flow-based programming check it out.

Elasticsearch + Kibana

I have been using the free-tier of elasticsearch for my NoSQL needs for a year now. It also has been highly useful for calculating similarity between dense vectors (or been alternatively using faiss). The only downside of elastic stack is the steep learning curve, and the pricing💸💸💸

Python library packaging tools

It is one thing to use and modify a python library…a whole another thing to create a python library from scratch and then dare to open-source it for all eyes to see 👀 & then maintain it 🙈 Being (still) a noobie all that I can say is that the following libraries helped me out so they deserve a mention.

  • Poetry
  • Dephell recently archived 😞 RIP 👼 Hope someone picks it up

I have also tried various cookie-cutters but none was to my liking… sure I can make my own but 😴

Last year was the start of my first(-ish) real open-source contributions. I’m still learning/trying to find time to write proper documentation and tests. So If you are using any of my libraries don’t be afraid to open issues to push me along 😬

Remarks

I like learning about new projects to make my life easier, allowing me to max laziness😴

As much as I want to go into more detail I have held back quite a bit & tried to keep it short. If you want to know more about one of these projects let me know and I can make a post about it. Share your beloved libraries in the comments.

Thanks to all the open-source contributers involved~

Resources to keep an eye on: - LF AI & Data Foundation Interactive Landscape: https://landscape.lfai.foundation/

Q&A

Questions & Contributions & Comments are welcome~

Follow me on Twitter to be notified of new posts~