Tag: programming

Search functionality in a small website like mine is usually arguably useless. I, for once, never even care to check whether a specific website offers it. I find a post that interests me via a search engine or aggregator, navigate to the page, read the post and then leave. However, I am not against local, serverless indexing and searching, even though most search engines provide site-specific searches. That is why I moved the search function of this website to a local, JavaScript-based implementation. How to do it? Read on.

In this post, I’m mirroring the Gaia Sky 3 tutorial I wrote for the official Gaia Sky documentation to use as a rough script for the workshop given in a splinter session of the 2021 DPAC consortium online meeting held on March 17 and 18, 2021. You can find the original page here.

It’s been a while since I last talked about new Gaia Sky releases. Today I’m doing a recap of the last four releases, starting with 3.0.0. This very verison came out with Gaia eDR3 on Dec 3, 2020. It was a big jump for Gaia Sky, as it introduced a plethora of new features and QOL improvements along with lots of bug fixes and little tweaks. This post goes over the latest versions from 3.0.0 to 3.0.3, and reflects on what they brought to the table.

Jump to the analysis for each of the versions directly:

• 3.0.0
• 3.0.1
• 3.0.2
• 3.0.3

In my re-implementation of the Gaia Sky level-of-detail (LOD) catalog generation in Rust I have been able to roughly halve the processing time, and, even though I do not have concrete numbers yet, everything points towards a drastic decrease in memory usage as well. In this project, I need to read a metric ton of gzipped csv Gaia catalog files, parse and process them into a functional in-memory catalog with cartesian positions, velocity vectors, RGB colors, etc. Then I need use them to generate an octree that represents the LOD structure, and finally write another metric ton of binary files back to disk. Using memory mapped files helps a lot in avoiding copies and speeding up the reading and writing operations; that’s something I tried out in the Java version and have come to also re-implement in Rust. Here’s the thing though: working with memory mapped files in Java is super straightforward. In Rust? Not so much. And the lack of available documentation and examples does not help. I was actually unable to find any working snippets with all the parts I needed, so I’m documenting it in this post in case someone else is in the same situation I was.

If you usually develop your software without an IDE, it may come in handy to be able to run a custom command or two whenever a file or a group of files in the file system is modified. This post discusses entr, a small event notify test runner which might just be what you need to fill an inconvenient gap in your mouseless development environment.

Lately I’ve taken an interest in the Rust programming language. I read the epub version of the Rust book over summer and found it quite well-written and overall interesting. Recently, I got myself a hard copy that sits on my desktop for quick queries, as I’ve been re-implementing the Gaia Sky catalog generation in Rust. The reasons for this are varied, but basically they come down to hardware. We need to load a humongous amount of stars (~1.6 billion) into memory, our old server, which had 2 TB of RAM, is being replaced by a newer, faster, and smaller (memory-wise, ~700 GB) machine in which we won’t be able to generate with the current Java implementation. But this is a story for another post. Today I’m here to document the completely mouseless Rust development environment I’ve been putting together as a sort of side project, to aid in my re-implementation.