Jekyll2024-01-26T16:10:20+00:00https://nithinbekal.com/feed.xmlNithin BekalNithin Bekal's blog about programming - Ruby, Rails, Vim, Elixir.Rake task for Jekyll stats2024-01-26T00:00:00+00:002024-01-26T00:00:00+00:00https://nithinbekal.com/posts/jekyll-stats<p>A few months ago,
I realized that I had
<a href="/posts/100-posts/">written 100 posts on this blog</a>.
This prompted me to add
a rake task to quickly look up
stats about this blog.
This is highly coupled to this blog’s theme,
but hopefully someone might be able to
convert this to work for their own blog.</p>
<p>The taks prints out
the number of posts,
the total number of words written
and number of posts by year.
Here’s the task from my <code class="language-plaintext highlighter-rouge">Rakefile</code>:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s2">"nokogiri"</span>
<span class="n">desc</span> <span class="s2">"Print stats about this blog"</span>
<span class="n">task</span> <span class="ss">:stats</span> <span class="k">do</span>
<span class="n">yearly_stats</span> <span class="o">=</span> <span class="no">Dir</span><span class="p">[</span><span class="s2">"_posts/*"</span><span class="p">]</span>
<span class="p">.</span><span class="nf">map</span> <span class="p">{</span> <span class="o">|</span><span class="n">filepath</span><span class="o">|</span> <span class="n">filepath</span><span class="p">.</span><span class="nf">delete_prefix</span><span class="p">(</span><span class="s2">"_posts/"</span><span class="p">).</span><span class="nf">split</span><span class="p">(</span><span class="s2">"-"</span><span class="p">).</span><span class="nf">first</span> <span class="p">}</span>
<span class="p">.</span><span class="nf">tally</span>
<span class="p">.</span><span class="nf">sort_by</span> <span class="p">{</span> <span class="o">|</span><span class="n">k</span><span class="p">,</span> <span class="n">_v</span><span class="o">|</span> <span class="n">k</span> <span class="p">}</span>
<span class="p">.</span><span class="nf">to_h</span>
<span class="n">total_posts</span> <span class="o">=</span> <span class="n">yearly_stats</span><span class="p">.</span><span class="nf">values</span><span class="p">.</span><span class="nf">sum</span><span class="p">(</span><span class="o">&</span><span class="ss">:to_i</span><span class="p">)</span>
<span class="n">total_words</span> <span class="o">=</span> <span class="no">Dir</span><span class="p">[</span><span class="s2">"_site/posts/*/index.html"</span><span class="p">]</span>
<span class="p">.</span><span class="nf">sum</span> <span class="p">{</span> <span class="o">|</span><span class="n">file</span><span class="o">|</span> <span class="n">word_count</span><span class="p">(</span><span class="n">file</span><span class="p">)</span> <span class="p">}</span>
<span class="nb">puts</span> <span class="o"><<~</span><span class="no">STATS</span><span class="sh">
Total posts: </span><span class="si">#{</span><span class="n">total_posts</span><span class="p">.</span><span class="nf">to_s</span><span class="p">.</span><span class="nf">rjust</span><span class="p">(</span><span class="mi">8</span><span class="p">)</span><span class="si">}</span><span class="sh">
Total words: </span><span class="si">#{</span><span class="n">number_to_human</span><span class="p">(</span><span class="n">total_words</span><span class="p">).</span><span class="nf">rjust</span><span class="p">(</span><span class="mi">8</span><span class="p">)</span><span class="si">}</span><span class="sh">
Yearly stats
</span><span class="no"> STATS</span>
<span class="n">yearly_stats</span><span class="p">.</span><span class="nf">each</span> <span class="k">do</span> <span class="o">|</span><span class="n">year</span><span class="p">,</span> <span class="n">count</span><span class="o">|</span>
<span class="nb">puts</span> <span class="s2">"</span><span class="si">#{</span><span class="n">year</span><span class="si">}</span><span class="se">\t</span><span class="si">#{</span><span class="n">count</span><span class="p">.</span><span class="nf">to_s</span><span class="p">.</span><span class="nf">rjust</span><span class="p">(</span><span class="mi">4</span><span class="p">)</span><span class="si">}</span><span class="se">\t</span><span class="si">#{</span><span class="s2">"|"</span> <span class="o">*</span> <span class="n">count</span><span class="si">}</span><span class="s2">"</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">word_count</span><span class="p">(</span><span class="n">file</span><span class="p">)</span>
<span class="no">File</span><span class="p">.</span><span class="nf">open</span><span class="p">(</span><span class="n">file</span><span class="p">)</span> <span class="p">{</span> <span class="o">|</span><span class="n">f</span><span class="o">|</span> <span class="no">Nokogiri</span><span class="o">::</span><span class="no">HTML</span><span class="p">(</span><span class="n">f</span><span class="p">)</span> <span class="p">}</span>
<span class="p">.</span><span class="nf">at_css</span><span class="p">(</span><span class="s2">".post .content"</span><span class="p">)</span>
<span class="p">.</span><span class="nf">text</span>
<span class="p">.</span><span class="nf">strip</span>
<span class="p">.</span><span class="nf">split</span>
<span class="p">.</span><span class="nf">count</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">number_to_human</span><span class="p">(</span><span class="n">n</span><span class="p">)</span>
<span class="n">n</span><span class="p">.</span><span class="nf">to_s</span>
<span class="p">.</span><span class="nf">split</span><span class="p">(</span><span class="s2">""</span><span class="p">)</span>
<span class="p">.</span><span class="nf">reverse</span>
<span class="p">.</span><span class="nf">each_slice</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
<span class="p">.</span><span class="nf">map</span><span class="p">(</span><span class="o">&</span><span class="ss">:join</span><span class="p">)</span>
<span class="p">.</span><span class="nf">join</span><span class="p">(</span><span class="s2">","</span><span class="p">)</span>
<span class="p">.</span><span class="nf">reverse</span>
<span class="k">end</span>
</code></pre></div></div>
<p>If you want to customize this
for your own blog,
there are a couple of things
that might be different for you:</p>
<ul>
<li>The <code class="language-plaintext highlighter-rouge">word_count</code> method has a reference
to the <code class="language-plaintext highlighter-rouge">".post .content"</code> DOM element,
which might be different in other blogs.</li>
<li>The permalink for my blog is <code class="language-plaintext highlighter-rouge">/posts/:title/</code>,
but you might have something different,
so you’ll need to chagne the <code class="language-plaintext highlighter-rouge">"posts/*"</code> everywhere
to match the pattern for your permalink.</li>
</ul>
<p>When I ran <code class="language-plaintext highlighter-rouge">rake stats</code> for this blog,
this is what I saw:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>Total posts: 110
Total words: 65,348
Yearly stats
2010 5 |||||
2011 4 ||||
2012 1 |
2013 1 |
2014 20 ||||||||||||||||||||
2015 32 ||||||||||||||||||||||||||||||||
2016 11 |||||||||||
2017 6 ||||||
2018 1 |
2019 4 ||||
2020 5 |||||
2021 1 |
2022 2 ||
2023 13 |||||||||||||
2024 4 ||||
</code></pre></div></div>
<p>Wow, 65,000 is a lot words.
That’s approaching a novel’s size!</p>A few months ago, I realized that I had written 100 posts on this blog. This prompted me to add a rake task to quickly look up stats about this blog. This is highly coupled to this blog’s theme, but hopefully someone might be able to convert this to work for their own blog.Numeric operations on value objects in Ruby2024-01-21T00:00:00+00:002024-01-21T00:00:00+00:00https://nithinbekal.com/posts/numeric-operations-value-objects<p>Here’s a little Ruby puzzle.
Imagine you have a <code class="language-plaintext highlighter-rouge">Value</code> class
that wraps a number,
and implements numeric operations.
<strong>How would you go about implementing the class
in a way that lets you do something like
<code class="language-plaintext highlighter-rouge">2 + Value.new(3)</code>
and returns <code class="language-plaintext highlighter-rouge">Value.new(5)</code>?</strong></p>
<p>To start with,
let’s describe what the <code class="language-plaintext highlighter-rouge">Value</code> class looks like:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Value</span>
<span class="nb">attr_reader</span> <span class="ss">:num</span>
<span class="k">def</span> <span class="nf">initialize</span><span class="p">(</span><span class="n">num</span><span class="p">)</span>
<span class="vi">@num</span> <span class="o">=</span> <span class="n">num</span>
<span class="k">end</span>
<span class="k">def</span> <span class="nf">+</span><span class="p">(</span><span class="n">other</span><span class="p">)</span>
<span class="no">Value</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="nb">self</span><span class="p">.</span><span class="nf">num</span> <span class="o">+</span> <span class="n">other</span><span class="p">.</span><span class="nf">num</span><span class="p">)</span>
<span class="k">end</span>
<span class="c1"># other operations: -, *, /</span>
<span class="k">end</span>
</code></pre></div></div>
<p>This will allow you do to things like this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="no">Value</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">2</span><span class="p">)</span> <span class="o">+</span> <span class="no">Value</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span> <span class="c1">#=> #<Value(5)</span>
</code></pre></div></div>
<p>Changing this class to accept a numeric value
on the right hand side
is pretty straightforward.
We can change the <code class="language-plaintext highlighter-rouge">+</code> method on <code class="language-plaintext highlighter-rouge">Value</code>
to wrap non-Value arguments as value objects.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">+</span><span class="p">(</span><span class="n">other</span><span class="p">)</span>
<span class="n">other</span> <span class="o">=</span> <span class="n">other</span><span class="p">.</span><span class="nf">is_a?</span><span class="p">(</span><span class="n">value</span><span class="p">)</span> <span class="p">?</span> <span class="n">other</span> <span class="p">:</span> <span class="no">Value</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">other</span><span class="p">)</span>
<span class="no">Value</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="nb">self</span><span class="p">.</span><span class="nf">num</span> <span class="o">+</span> <span class="n">other</span><span class="p">.</span><span class="nf">num</span><span class="p">)</span>
<span class="k">end</span>
<span class="no">Value</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span> <span class="o">+</span> <span class="mi">2</span> <span class="c1">#=> #<Value(5)></span>
</code></pre></div></div>
<p>However, this doesn’t work in reverse.
This is because
when we write <code class="language-plaintext highlighter-rouge">2 + something</code>,
Ruby calls the <code class="language-plaintext highlighter-rouge">+</code> method on the integer,
so this is equivalent to <code class="language-plaintext highlighter-rouge">2.+(something)</code>.
<code class="language-plaintext highlighter-rouge">Integer#+</code> doesn’t know how to handle <code class="language-plaintext highlighter-rouge">Value</code>s
so we’ll see this exception:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="mi">2</span> <span class="o">+</span> <span class="no">Value</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span>
<span class="c1">#=> 🔥 Value can't be coerced into Integer (TypeError)</span>
</code></pre></div></div>
<p>The first idea that came to my mind
when thinking of a way to do this in reverse
was to monkeypatch <code class="language-plaintext highlighter-rouge">Numeric</code>.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Numeric</span>
<span class="k">def</span> <span class="nf">+</span><span class="p">(</span><span class="n">other</span><span class="p">)</span>
<span class="k">if</span> <span class="n">other</span><span class="p">.</span><span class="nf">is_a?</span><span class="p">(</span><span class="no">Value</span><span class="p">)</span>
<span class="no">Value</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="nb">self</span><span class="p">)</span> <span class="o">+</span> <span class="n">other</span>
<span class="k">else</span>
<span class="k">super</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Monkeypatching core classes
is a terrible idea,
so let’s look for another way.
The error message we saw earlier
(“Value can’t be coerced into Integer”)
gives us a clue about what to do.
There must be a way to coerce <code class="language-plaintext highlighter-rouge">Value</code> into <code class="language-plaintext highlighter-rouge">Integer</code>.</p>
<p>Can we define <code class="language-plaintext highlighter-rouge">to_i</code> on <code class="language-plaintext highlighter-rouge">Value</code>?
Nope, same error.
Defining <code class="language-plaintext highlighter-rouge">to_f</code> doesn’t help either.</p>
<p>Next let’s look at
<a href="https://github.com/ruby/ruby/blob/34315510d34543cf14fe0ac9e8adb1d86b5beebf/numeric.c#L3990">how <code class="language-plaintext highlighter-rouge">+</code> is implemented on Integer</a>.
When you perform an arithmetic operation on an integer
and it doesn’t know what to do with the argument,
it calls <code class="language-plaintext highlighter-rouge">coerce</code> on that object.
This method should return a 2-element array
containing the objects
modified into compatible types.
It then calls <code class="language-plaintext highlighter-rouge">+</code> on the resulting objects.</p>
<p>We can now define a <code class="language-plaintext highlighter-rouge">Value#coerce</code> method,
that returns a tuple.
Ther first element here should be the number
wrapped into a <code class="language-plaintext highlighter-rouge">Value</code> object,
and the second should be the value object being added to it.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">def</span> <span class="nf">coerce</span><span class="p">(</span><span class="n">n</span><span class="p">)</span>
<span class="p">[</span><span class="no">Value</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">n</span><span class="p">),</span> <span class="nb">self</span><span class="p">]</span>
<span class="k">end</span>
<span class="c1"># And now this works!</span>
<span class="mi">2</span> <span class="o">+</span> <span class="no">Value</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span> <span class="c1">#=> <Value(5)></span>
</code></pre></div></div>
<p>There’s one gotcha to note here:
it might be tempting to return <code class="language-plaintext highlighter-rouge">[self, n]</code> here
because <code class="language-plaintext highlighter-rouge">self.+(n)</code> can handle both numbers and <code class="language-plaintext highlighter-rouge">Value</code>s.
However, the coerce method is used
for all numeric operations,
including <code class="language-plaintext highlighter-rouge">-</code> and <code class="language-plaintext highlighter-rouge">/</code>
where the order of arguments matters
(<code class="language-plaintext highlighter-rouge">x/y</code> is not the same as <code class="language-plaintext highlighter-rouge">y/x</code>),
so it’s important to return the values
in the same order.</p>
<h2 id="python">Python</h2>
<p>I came across this problem
while working through
<a href="https://karpathy.ai/zero-to-hero.html">Andrej Karpathy’s neural networks course</a>
and trying to reimplement
his <a href="https://github.com/karpathy/micrograd">micrograd</a> package written in Python.</p>
<p>Python takes a different approach to this,
and lets you define magic methods
for numeric operations.
For a similar <code class="language-plaintext highlighter-rouge">Value</code> class
written in Python,
you would do something like this:</p>
<div class="language-python highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="k">class</span> <span class="nc">Value</span><span class="p">:</span>
<span class="c1"># ...
</span>
<span class="c1"># Value(2) + Value(3)
</span> <span class="k">def</span> <span class="nf">__add__</span><span class="p">(</span><span class="n">other</span><span class="p">):</span>
<span class="n">Value</span><span class="p">(</span><span class="bp">self</span><span class="p">.</span><span class="n">num</span> <span class="o">+</span> <span class="n">other</span><span class="p">.</span><span class="n">num</span><span class="p">)</span>
<span class="c1"># 2 + Value(3)
</span> <span class="k">def</span> <span class="nf">__radd__</span><span class="p">(</span><span class="n">other</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span> <span class="o">+</span> <span class="n">other</span>
<span class="mi">2</span> <span class="o">+</span> <span class="n">Value</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span> <span class="c1">#=> Value(5)
</span></code></pre></div></div>
<p>I prefer the Ruby approach,
because you don’t need to implement these magic methods
for all numeric operations.
However, the explicitness of magic methods
means that we don’t need to worry
about returning the operands in an incorrect order.</p>
<p>If you want to compare the <code class="language-plaintext highlighter-rouge">Value</code> class in the two languages,
you can see them here:</p>
<ul>
<li><a href="https://github.com/karpathy/micrograd/blob/master/micrograd/engine.py">Original Python implementation</a></li>
<li><a href="https://github.com/nithinbekal/micrograd/blob/main/lib/micrograd/value.rb">Ruby implementation</a></li>
</ul>
<p>I looked around to find real life usage of <code class="language-plaintext highlighter-rouge">coerce</code>,
and found that the
<a href="https://github.com/RubyMoney/money/blob/092ecc7a711263208ae12fd62e7538f3d7a6348a/lib/money/money/arithmetic.rb#L326">the money gem uses coerce</a>
to allow adding numbers and <code class="language-plaintext highlighter-rouge">Money</code> objects.</p>Here’s a little Ruby puzzle. Imagine you have a Value class that wraps a number, and implements numeric operations. How would you go about implementing the class in a way that lets you do something like 2 + Value.new(3) and returns Value.new(5)?Long exposure images from videos2024-01-15T00:00:00+00:002024-01-15T00:00:00+00:00https://nithinbekal.com/posts/long-exposure-video<p>Last summer,
I was at the Chaudiere Falls in Ottawa.
A long exposure photo
would have been perfect
to capture the turbulent cascades of the falls.
Unfortunately,
I didn’t have an ND filter with me.
Instead, I decided to capture a short video
on my phone
to see if I could
convert its frames into a long exposure.</p>
<p>There’s probably better ways to do this
using Photoshop or GIMP,
but I wanted to do this
using a simple command line script
that takes a video
and generates an image.</p>
<h2 id="steps-to-convert-a-video-to-a-long-exposure-image">Steps to convert a video to a long exposure image</h2>
<p>This will involve three steps:</p>
<ol>
<li><strong>Extract frames from the video</strong>
<ul>
<li><code class="language-plaintext highlighter-rouge">ffmpeg</code> can do this for us</li>
</ul>
</li>
<li><strong>Align the images</strong>
<ul>
<li>It was a handheld video,
and the frames won’t be aligned,
so we’ll need to use <code class="language-plaintext highlighter-rouge">align_image_stack</code>
to align them</li>
</ul>
</li>
<li><strong>Stack the images</strong>
<ul>
<li><code class="language-plaintext highlighter-rouge">imagemagick</code> can be used for this</li>
</ul>
</li>
</ol>
<h3 id="installing-the-tools">Installing the tools</h3>
<p>Both <code class="language-plaintext highlighter-rouge">imagemagick</code> and <code class="language-plaintext highlighter-rouge">ffmpeg</code>
are available via homebrew on Mac,
but <code class="language-plaintext highlighter-rouge">align_image_stack</code>
is part of a panorama stitching tool called Hugin,
which can be installed via homebrew-cask.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>brew <span class="nb">install </span>ffmpeg imagemagick
brew <span class="nb">install </span>hugin <span class="nt">--cask</span>
</code></pre></div></div>
<h3 id="extracting-frames">Extracting frames</h3>
<p>The first step is to take the video and extract frames from it using ffmpeg.
The <code class="language-plaintext highlighter-rouge">-r</code> flag indicates that this will extract 4 frames for each second in the video.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>ffmpeg <span class="nt">-i</span> input.mp4 <span class="nt">-r</span> 4 frames/%04d.png
</code></pre></div></div>
<p>This will create files named <code class="language-plaintext highlighter-rouge">frames/*.png</code>,
with the files numbered
(<code class="language-plaintext highlighter-rouge">0001.png</code>, <code class="language-plaintext highlighter-rouge">0002.png</code> and so on).</p>
<h3 id="aligning-the-images">Aligning the images</h3>
<p>I didn’t use a tripod to stabilize the video,
so the images will not be perfectly aligned.
To align them,
we will use the <code class="language-plaintext highlighter-rouge">align_image_stack</code> tool from the Hugin tool.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>/Applications/Hugin/tools_mac/align_image_stack frames/<span class="k">*</span>.png <span class="nt">-a</span> aligned/a <span class="nt">-v</span>
</code></pre></div></div>
<h3 id="stacking-the-images">Stacking the images</h3>
<p>Now we have the aligned images as <code class="language-plaintext highlighter-rouge">aligned/*.tif</code>.
The final step is to average these images using imagemagick.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>convert aligned/<span class="k">*</span>.tif <span class="nt">-average</span> final.jpg
</code></pre></div></div>
<p>Here’s the full
<a href="https://gist.github.com/nithinbekal/b58260c342b92d9b62615a2ccb1e926b">bash script for converting a video to a long exposure image</a></p>
<h2 id="the-result">The result</h2>
<p>For the video I took over the weekend,
the end result wasn’t too great.
It’s not very sharp,
and there’s some clear vignetting
because of the handheld capture.
Here’s the final photo:</p>
<p><img loading="lazy" alt="Chaudiere Falls long exposure photo" src="https://s3.amazonaws.com/nithinbekal.com/blog/long-exposure-video/chaudiere-falls-long-exposure.webp" /></p>
<p>Here’s the video
from which it was generated.</p>
<div class="youtube-embed-container">
<iframe width="640" height="390" src="https://www.youtube.com/embed/p-x_e7nKseY" frameborder="0" allowfullscreen=""></iframe>
</div>
<p>I also had a few videos
that I took last year at Yellowknife
with the intention of trying this experiment,
and I promptly forgot about them until now.
Here are a couple of them
taken during a hike to Cameron Falls near Yellowknife.</p>
<p><img loading="lazy" alt="Cameron Falls Yellowknife long exposure photo" src="https://s3.amazonaws.com/nithinbekal.com/blog/long-exposure-video/yellowknife-cameron-falls-long-exposure.webp" /></p>
<p>The image on the left
was from a tripod-stabilized video,
so looks a lot sharper.
The one on the right
is a handheld capture,
so it’s not quite the same sharpness.</p>
<h2 id="conclusion">Conclusion</h2>
<p>Overall, the quality of the photos isn’t too great.
A phone mounted on a small tripod,
along with a simple
<a href="https://amzn.to/3S26kqT">phone lens clip</a>
and a cheap <a href="https://amzn.to/3ShOod7">ND1000 filter</a>
could have produced much better photos.</p>
<p>Also, there are apps for the phone these days
which can do all of this on the phone.
On an Iphone,
you don’t even need to install
a third-party app.</p>
<p>It was still a fun expermient,
but the only reason you’d do this
is if you don’t have a tripod with you.
One thing I might try next time
is using an intervalometer app
to take multiple photos
and merge them together.
This will provide
higher resolution images
than we get from even a 4K video.</p>
<h3 id="links">Links</h3>
<ul>
<li><a href="https://gist.github.com/nithinbekal/b58260c342b92d9b62615a2ccb1e926b">Bash script to convert a video to a long exposure image</a></li>
<li>I’m <a href="https://www.instagram.com/nithinbekal">@nithinbekal on instagram</a>
if you wanted to see more of my photos.
Unfortunately,
I’ve yet to post a long exposure photo there! :)</li>
</ul>Last summer, I was at the Chaudiere Falls in Ottawa. A long exposure photo would have been perfect to capture the turbulent cascades of the falls. Unfortunately, I didn’t have an ND filter with me. Instead, I decided to capture a short video on my phone to see if I could convert its frames into a long exposure.Favorite books of 20232024-01-13T00:00:00+00:002024-01-13T00:00:00+00:00https://nithinbekal.com/posts/books-2023<p>28 books.
8900 pages.
That’s more than I was expecting
to read this year.
I also just realized that
my <a href="/posts/favorite-books-2013/">first favorite books post</a>
was written almost exactly 10 years ago!</p>
<h2 id="fiction">Fiction</h2>
<p>Despite not being much of a graphic novel reader,
two of this year’s favorites of mine
are graphic novels.
Aside from those,
this year’s fiction reading
was dominated by fantasy novels.</p>
<p><strong><a href="https://amzn.to/3S4je7U">Maus</a> (Art Spiegelman)</strong></p>
<p>This graphic novel tells the story of the author’s father, Vladek,
a Jewish survivor of the Holocaust.
The story is presented through the lens of conversations
that Art has with Vladek in the modern day.
Although primarily about Vladek’s experiences in a concentration camp,
it also shows the complexity of the father-son relationship in the present day.
Be warned, though -
this book can be a gut wrenchingly painful read.</p>
<p><strong><a href="https://amzn.to/3HgyV6J">Season of Mists</a> (Neil Gaiman)</strong></p>
<p>Another graphic novel,
and this was a thoroughly enjoyable addition to the Sandman series.
I found the previous books in this series to be a bit disjointed,
but here there’s a single story
that constantly held my attention.</p>
<p><strong><a href="https://amzn.to/3Hgz4ah">Gardens of the Moon</a> (Steven Eriksen)</strong></p>
<p>The epic fantasy novel drops the reader right into the action
without any background on the story or the characters.
I’ve tried reading this book in the past,
and given up after a few pages.
This time I powered through the first 100 pages
until I could start making sense of the story,
and was rewarded with beautifully written prose
and incredible worldbuilding.</p>
<p><strong><a href="https://amzn.to/3tSoy64">The Secret Projects</a> novels by Brandon Sanderson</strong></p>
<p>In 2022, Brandon Sanderson announced
that he had written 4 novels in secret during the pandemic,
in addition to all the other novels he published.
The Kickstarter for publishing these books
broke all records and raised over $40 million.
3 of those books made it onto this list:</p>
<ul>
<li>
<p><strong><a href="https://amzn.to/3NYAkCA">Tress of the Emerald Sea</a></strong>:
Beautiful fairy tale adventure
inspired by The Princess Bride.
But here the girl that goes on an adventure
to rescue her man.</p>
</li>
<li>
<p><strong><a href="https://amzn.to/3ScrB2h">Yumi and the Nightmare Painter</a></strong>:
Sanderson’s ability to write action is in full display here.
One of the protagonists fights nightmares that manifest in the real world
by painting them.
You wouldn’t expect that to be the formula
for edge-of-the-seat excitement,
but somehow he manages it.
Had a genuinely surprising revelation at the end.</p>
</li>
<li>
<p><strong><a href="https://amzn.to/4aOws17">The Sunlit Man</a></strong>:
Set in a world where sunlight instantly incinerates everything,
people have to live in airborne cities
that constantly fly to keep them on the dark side.
An offworlder arrives and challenges
the tyrant that rules that world.
While it has connections
to Sanderson’s Stormlight Archives series,
it works really well as a standalone story.</p>
</li>
</ul>
<h2 id="non-fiction">Non fiction</h2>
<p><strong><a href="https://amzn.to/3RVhnlI">Pale Blue Dot</a> (Carl Sagan)</strong></p>
<p>One of the best books on space exploration out there.
The title refers to the photo of the Earth taken
by the Voyager space probe from 6 billion km away.
In it, our planet spans a single pixel
amidst the vastness of space,
and Sagan uses that to reflect
on our place in the cosmos.
This beautiful passage is a perfect example
of what to expect from this book.</p>
<blockquote>
<p>Look again at that dot. That’s here. That’s home. That’s us. On it everyone you love, everyone you know, everyone you ever heard of, every human being who ever was, lived out their lives. The aggregate of our joy and suffering, thousands of confident religions, ideologies, and economic doctrines, every hunter and forager, every hero and coward, every creator and destroyer of civilization, every king and peasant, every young couple in love, every mother and father, hopeful child, inventor and explorer, every teacher of morals, every corrupt politician, every “superstar,” every “supreme leader,” every saint and sinner in the history of our species lived there–on a mote of dust suspended in a sunbeam.</p>
</blockquote>
<p><strong><a href="https://amzn.to/3RTBikV">Staff Engineer</a> (Will Larson)</strong></p>
<p>There are a lot of books for people taking the engineering management track,
but not many for those of us on the Staff track.
This book has become the canonical book in the space.
There’s a lot of practical wisdom in the first half of the book.
The second half contains interviews with Staff+ engineers from many different companies,
providing insights into the many different roles
that Staff+ engineers play.</p>
<p><strong><a href="https://amzn.to/3O0kYxG">An Elegant Puzzle</a> (Will Larson)</strong></p>
<p>Another great book from Larson
to complement the Staff Engineer book.
This one is written
for engineering managers.
However, part of Staff engineers’ role
is to be a partner to an engineering manager,
and I found this a great way to understand their perspective.</p>
<p><strong><a href="https://amzn.to/3TYJHWN">Don’t Make Me Think</a> (Steve Krug)</strong></p>
<p>If <a href="https://amzn.to/3RPakLe">The Design of Everyday Things</a>
were to be written from the perspective of web usability,
this book would be pretty close to what it should look like.
In this short and engaging book,
Krug walks you through how to think about usability.
A decade is a long time for the Web and UX,
but his advice holds up pretty well,
and will likely do so for another decade.</p>
<p><strong><a href="https://amzn.to/3tHXJS9">Ruby Under a Microscope</a> (Pat Shaugnessy)</strong></p>
<p>If you’re a Ruby programmer
looking to understand the internals of the CRuby implementation,
this is the book for you.
With clear illustrations,
it walks you through different aspects of the Ruby VM,
while also helping you get
a deeper understanding of the Ruby object model.</p>
<h2 id="parenting">Parenting</h2>
<p>This year is my first as a parent,
so a good chunk of my reading
was related to parenting.</p>
<p><strong><a href="https://amzn.to/41WEErW">Bringing Up Bebe</a> (Pamela Druckerman)</strong></p>
<p>Druckerman is an American journalist
raising children in France,
putting her in a great position to write
this entertaining comparison of French and American parenting styles.
The writing is witty and engaging,
and I found myself reading chapters
about things that won’t ever affect me.
The contrast with French culture also made me realize
how crazy parenting in North America seems by comparison.</p>
<p><strong><a href="https://amzn.to/41WEMrq">Precious Little Sleep</a> (Alexis Dubief)</strong></p>
<p>Sleep training a baby can be life changing for the parents,
going from constant sleep deprivation
to getting decent amount of sleep every night.
This book does a great job
organizing all the information out there about the subject.
However, I found the writing style annoying.
The content comes from a blog
the author used to write,
so it feels like reading a random blog on the internet.
If all the fluff had been edited out,
this would have been 50 pages shorter
and way more easy to read.
Still a very worthwhile read.</p>
<p><strong><a href="https://amzn.to/48O25G8">The Birth Partner</a> (Penny Simkin)</strong></p>
<p>This book is a guide for the support person during the birth.
This has great advice on how to deal with
the many difficult decisions you’ll have to make
in the hospital during the birth.
I picked this up a few days before the birth of our child,
and only managed to finish a third of the book when she was born,
and regretted not starting it sooner.</p>
<h2 id="2024">2024</h2>
<p>I’ve set myself a reading goal of 30 this year,
which is slightly more than the 28 I finished this year.
That’s going to be tough if I’m going to continue
with the Malazan series I started last year
with 9 more 1000-page tomes to go!</p>
<p>I was also hoping to make progress last year
on the series I’d already started earlier,
like Dune, Hyperion Expanse and Murderbot Diaries,
but never made any progress there.
I’m hoping to finish at least the Murderbot series this year.</p>
<div class="note">
<p>
This is part of my
<em>Favorite Books of the Year</em>
series.
You can find the other posts below,
or
<a href="https://www.goodreads.com/user/show/1059476-nithin-bekal">follow me on Goodreads</a>.
</p>
<p>
<a href="/posts/books-2023/">2023</a>
<a href="/posts/books-2022/">2022</a>
<a href="/posts/books-2021/">2021</a>
<a href="/posts/books-2020/">2020</a>
<a href="/posts/books-2019/">2019</a>
<a href="/posts/books-2018/">2018</a>
<a href="/posts/favorite-books-2017/">2017</a>
<a href="/posts/favorite-books-2016/">2016</a>
<a href="/posts/favorite-books-2015/">2015</a>
<a href="/posts/favorite-books-2014/">2014</a>
<a href="/posts/favorite-books-2013/">2013</a>
</p>
</div>28 books. 8900 pages. That’s more than I was expecting to read this year. I also just realized that my first favorite books post was written almost exactly 10 years ago!What’s new in Ruby 3.32023-12-17T00:00:00+00:002023-12-17T00:00:00+00:00https://nithinbekal.com/posts/ruby-3-3<p>Every year on Christmas day,
the Ruby core team releases
a new version of Ruby.
This year will likely be no different,
and we can expect Ruby 3.3 next week.</p>
<p>This year,
the primary focus of the release
is performance and developer experience.
There aren’t as many
major changes to the language features
as in previous versions,
so this post is going to be shorter than usual.
Here are some of
the highlights of the release:</p>
<h3 id="yjit">YJIT</h3>
<p>YJIT, Ruby’s JIT compiler,
has seen some incredible advances
in the past couple of years.
This year,
it has
<a href="https://railsatscale.com/2023-12-04-ruby-3-3-s-yjit-faster-while-using-less-memory/">continued getting faster, while also consuming less memory</a>.</p>
<p>Companies like
<a href="https://dev.37signals.com/yjit-is-fast/">Basecamp</a>
and
<a href="https://railsatscale.com/2023-09-18-ruby-3-3-s-yjit-runs-shopify-s-production-code-15-faster/">Shopify</a>
are already running
with preview releases of 3.3 in production
and have seen around 15% improvement
in average response times,
compared to 3.3.0 without YJIT.
In fact,
YJIT has been so effective
at speeding up Rails apps
that it will be
<a href="https://github.com/rails/rails/pull/49947">enabled by default</a>
on newly generated Rails apps
if you are on Ruby 3.3.</p>
<p>A new method,
<code class="language-plaintext highlighter-rouge">RubyVM::YJIT.enable</code>
has been introduced,
which allows you to enable YJIT at runtime.
This can be useful
if you want to load the app quickly
and then enable YJIT
after it has booted.
This is what Rails uses to enable JIT
in an initializer.</p>
<h3 id="rjit">RJIT</h3>
<p>An experimental JIT compiler called RJIT
has been introduced.
This is written in pure Ruby,
and replaces MJIT,
which was an alternative JIT
that was available in Ruby 3.2.
RJIT uses a lot more memory than YJIT,
and exists only for experimentation.
In production,
you should always use YJIT.</p>
<p><em>(Update:
<a href="https://k0kubun.medium.com/rjit-a-pure-ruby-jit-for-ruby-f4084f0765">Here’s an article</a>
by the creator of RJIT
about why it was added to Ruby.)</em></p>
<h3 id="irb">IRB</h3>
<p>After using <code class="language-plaintext highlighter-rouge">pry</code> and <code class="language-plaintext highlighter-rouge">byebug</code>
as my preferred debugging tools for years,
I switched to using <a href="https://github.com/ruby/irb#usage">IRB</a>
for all my debugging
in the past few months.
With this release,
IRB, and its integration
with the builtin <a href="https://github.com/ruby/debug#how-to-use"><code class="language-plaintext highlighter-rouge">debug</code> gem</a>
has improved so much
that <code class="language-plaintext highlighter-rouge">pry</code> and <code class="language-plaintext highlighter-rouge">byebug</code>
are no longer necessary
for debugging your code.</p>
<h3 id="range">Range</h3>
<p>There are a couple of changes to the <code class="language-plaintext highlighter-rouge">Range</code> class.
An <code class="language-plaintext highlighter-rouge">overlap?</code> method has been added:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="mi">3</span><span class="p">).</span><span class="nf">overlap?</span><span class="p">(</span><span class="mi">3</span><span class="o">..</span><span class="mi">5</span><span class="p">)</span> <span class="c1"># true</span>
<span class="p">(</span><span class="mi">1</span><span class="o">..</span><span class="mi">3</span><span class="p">).</span><span class="nf">overlap?</span><span class="p">(</span><span class="mi">4</span><span class="o">..</span><span class="mi">6</span><span class="p">)</span> <span class="c1"># false</span>
</code></pre></div></div>
<p>You can also call <code class="language-plaintext highlighter-rouge">reverse_each</code>
on begin-less ranges now:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">(</span><span class="o">..</span><span class="mi">10</span><span class="p">).</span><span class="nf">reverse_each</span><span class="p">.</span><span class="nf">take</span><span class="p">(</span><span class="mi">3</span><span class="p">)</span> <span class="c1">#=> [10, 9, 8]</span>
</code></pre></div></div>
<h3 id="prism-parser">Prism parser</h3>
<p>A new parser called <a href="https://github.com/ruby/prism">Prism</a>
has been introduced as a default gem.
It is more error tolerant
and can be used as a replacement for ripper.
This is portable across Ruby implementations,
and is currently being integrated into
MRI, JRuby, TruffleRuby and Sorbet.</p>
<p>With prism,
you can parse code using
<code class="language-plaintext highlighter-rouge">Prism.parse</code>
and get a result that looks like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>> Prism.parse("1 + 2")
#<Prism::ParseResult:0x000000010f5518c8
@comments=[],
@data_loc=nil,
@errors=[],
@magic_comments=[],
@source=#<Prism::Source:0x000000012aa77530 @offsets=[0], @source="1 + 2", @start_line=1>,
@value=
@ ProgramNode (location: (1,0)-(1,5))
├── locals: []
└── statements:
@ StatementsNode (location: (1,0)-(1,5))
└── body: (length: 1)
└── @ CallNode (location: (1,0)-(1,5))
├── flags: ∅
├── receiver:
│ @ IntegerNode (location: (1,0)-(1,1))
│ └── flags: decimal
├── call_operator_loc: ∅
├── name: :+
├── message_loc: (1,2)-(1,3) = "+"
├── opening_loc: ∅
├── arguments:
│ @ ArgumentsNode (location: (1,4)-(1,5))
│ ├── flags: ∅
│ └── arguments: (length: 1)
│ └── @ IntegerNode (location: (1,4)-(1,5))
│ └── flags: decimal
├── closing_loc: ∅
└── block: ∅,
@warnings=[]>
</code></pre></div></div>
<h3 id="mn-thread-scheduler">M:N thread scheduler</h3>
<p>A new M:N thread scheduler has been introduced
to improve thread and ractor performance.
Here, M is the number of ractors
and N is the number of native threads
(equal to number of CPU cores, for instance).
This allows us to create a large number of ractors
without having to pay the overhead
of creating a native thread for each one of them.</p>
<h3 id="other-changes">Other changes</h3>
<ul>
<li>In Ruby 3.4,
there is a <a href="https://bugs.ruby-lang.org/issues/18980">proposal</a>
to use <code class="language-plaintext highlighter-rouge">it</code> as a reference
to the first argument to a block.
Calling a method called <code class="language-plaintext highlighter-rouge">it</code> without args
has been deprecated
and will show a warning.</li>
<li>Bison has been replaced with
<a href="https://github.com/yui-knk/lrama">Lrama parser generator</a>
to improve maintainability.</li>
<li>There have been more optimizations
to the garbage collector
to further improve performance.</li>
</ul>
<h3 id="more-reading">More reading</h3>
<p>With these posts,
I try to highlight some of the changes
that I found most interesting,
and often skip over features I might not use.
If you’re looking for a more comprehensive look at the release,
I highly recommend looking at the
<a href="https://www.ruby-lang.org/en/news/2023/12/25/ruby-3-3-0-released/">release announcement</a>,
<a href="https://github.com/ruby/ruby/blob/master/doc/NEWS/NEWS-3.3.0.md">changelog</a>,
and the excellent
<a href="https://rubyreferences.github.io/rubychanges/3.3.html">Ruby References website</a>
which covers all the new features in detail
with lots of examples.</p>
<div class="note">
<p>
This article is part of the <em>What's New in Ruby</em> series.
To read about a different version of Ruby,
pick the version here:
</p>
<p>
<a href="/posts/ruby-2-3-features/">2.3</a>
<a href="/posts/ruby-2-4-features/">2.4</a>
<a href="/posts/ruby-2-5-features/">2.5</a>
<a href="/posts/ruby-2-6/">2.6</a>
<a href="/posts/ruby-2-7/">2.7</a>
<a href="/posts/ruby-3-0/">3.0</a>
<a href="/posts/ruby-3-1/">3.1</a>
<a href="/posts/ruby-3-2/">3.2</a>
<a href="/posts/ruby-3-3/">3.3</a>
</p>
</div>Every year on Christmas day, the Ruby core team releases a new version of Ruby. This year will likely be no different, and we can expect Ruby 3.3 next week.15 years of programming2023-10-16T00:00:00+00:002023-10-16T00:00:00+00:00https://nithinbekal.com/posts/15-years-of-programming<p>Exactly 15 years ago, on this day,
I started my first job as a programmer.
Technically I’ve written code for a bit longer than that,
but this was the first time actually being paid to do it.
It’s not a particularly long career,
but I’ve been reflecting on how much
things have changed over that time.</p>
<h3 id="hardware">Hardware</h3>
<p>The computer I learned to program on
had a Pentium III
with 128MB of RAM.
The first real work computer I used
had similar specs.
The phone in my pocket today
probably has more computing power.</p>
<p>At a couple of places that I worked,
we worked using thin clients connected to remote servers.
I’ve been coding on a cloud dev environment
using an M1 Macbook Pro today.
That’s a ridiculously powerful machine
that’s essentially working as a thin client!</p>
<h3 id="ruby">Ruby</h3>
<p>I’ve used Ruby and Rails
for about 12 of those 15 years.
The first app I shipped
was written using Ruby 1.8 and Rails 2.3.
Bundler and gemfiles were still not a thing
until a few months later.
I got to see 13 major releases of Ruby since then,
and it has become
almost an order of magnitude faster.</p>
<h3 id="editors">Editors</h3>
<p>In 2009,
I was using Netbeans
because it was a proper Rails IDE.
For the past 9 years,
I’ve been using Vim,
and it can probably do more “IDE” stuff
than Netbeans did back then.</p>
<p>Notepad++ was the first editor I used regularly.
Since then,
I’ve gone through Sublime Text,
Textpad,
Redcar (an editor written in Ruby),
Atom,
and VS Code
(which I still use when pair programming).</p>
<p>Textpad was an interesting one.
The place where I worked
<em>didn’t have an approved text editor</em>,
despite 100s of devs working there.
Someone managed to get a copy of Textpad
through the company firewall,
and that became almost everybody’s editor.
There were a couple of people
who still chose to code on Windows Notepad, though!</p>
<h3 id="version-control">Version control</h3>
<p>Subversion was the version control system
for my first project.
Git had been around for a while by then,
but Github was still a new startup.
However, our choice of subversion was mainly because
it had better integration with Netbeans,
which our team used at the time.</p>
<p>The weirdest version control system I used
was an Excel spreadsheet.
At that place,
we would email the “version control team”
a list of files that we wanted to work on,
and they would put our name
next to the name of the file in the spreadsheet.
There was an actual VCS somewhere
(Visual Source Pro),
but we programmers weren’t trusted with using it.</p>
<h3 id="deployment">Deployment</h3>
<p>The first app I deployed
was to one of those shared hosting providers.
Deploying the app was ridiculously finicky.
I don’t remember the details,
but it involved rsync-ing the code,
and then ssh-ing in to restart the server
(and praying that things work!).</p>
<p>At the next place,
things were even worse.
We had manual deploys once a week.
Every Wednesday at 2am,
this guy would compile exe files,
and use Remote Desktop to copy them into the server.
If you didn’s send in your code by the previous afternoon,
you’d have to wait till next Wednesday,
unless you were putting out serious fires.</p>
<p>By 2012,
I was working on a Rails codebase once again,
and we set up Capistrano
so we could run <code class="language-plaintext highlighter-rouge">cap deloy</code> locally.
This seemed like a radical idea at the time,
but all of this is such a far cry
from the niceties that we enjoy today,
like CI pipelines or Heroku or Github Actions.</p>
<h3 id="wrapping-up">Wrapping up</h3>
<p>When I look back over the 15 years,
the improvement in tooling that we use
has been incredible.
I couldn’t even have imagined
tools like GPT or Copilot back then!</p>
<p>Each passing year
has also made it painfully obvious
how little I know about programming
and how much there is left to learn.
I hope I’ll still be enjoying this profession
in another 15 years time!</p>Exactly 15 years ago, on this day, I started my first job as a programmer. Technically I’ve written code for a bit longer than that, but this was the first time actually being paid to do it. It’s not a particularly long career, but I’ve been reflecting on how much things have changed over that time.Ruby’s bang methods - Handle with care!2023-08-08T00:00:00+00:002023-08-08T00:00:00+00:00https://nithinbekal.com/posts/ruby-bang-methods<p>In Ruby,
method names can be suffixed with <code class="language-plaintext highlighter-rouge">!</code>.
These are often called “bang” methods.
This is used to indicate
that it is a dangerous counterpart
of the non-bang version of the method.</p>
<p>I had always thought of “dangerous”
to mean that the method
mutates the underlying object
and returns that object.
However,
these methods can differ
from their non-bang versions
in other subtle ways.
This tripped me up recently.</p>
<h2 id="the-case-of-arrayreject">The case of <code class="language-plaintext highlighter-rouge">Array#reject!</code></h2>
<p>I was modifying some code recently,
and was surprised by how <code class="language-plaintext highlighter-rouge">Array#reject!</code> works.</p>
<p><code class="language-plaintext highlighter-rouge">Array#reject</code> - the non-bang version -
returns a new array
after removing elements for which
the block evaluates to a truthy value.
I had expected <code class="language-plaintext highlighter-rouge">reject!</code>
to work similarly,
with the exception that
it returns the mutated receiver.</p>
<p>Let’s look at an example
comparing the two methods,
when there is something to reject:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">].</span><span class="nf">reject</span><span class="p">(</span><span class="o">&</span><span class="ss">:even?</span><span class="p">)</span> <span class="c1">#=> [1, 3]</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">].</span><span class="nf">reject!</span><span class="p">(</span><span class="o">&</span><span class="ss">:even?</span><span class="p">)</span> <span class="c1">#=> [1, 3]</span>
</code></pre></div></div>
<p>Both of them returned an array.
Now, let’s try changing the input
to only contain odd numbers.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">].</span><span class="nf">reject</span><span class="p">(</span><span class="o">&</span><span class="ss">:even?</span><span class="p">)</span> <span class="c1">#=> [1, 3, 5]</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">].</span><span class="nf">reject!</span><span class="p">(</span><span class="o">&</span><span class="ss">:even?</span><span class="p">)</span> <span class="c1">#=> nil 💥</span>
</code></pre></div></div>
<p>Turns out <code class="language-plaintext highlighter-rouge">reject!</code> returns <code class="language-plaintext highlighter-rouge">nil</code>
if there’s nothing to reject.
This is how a lot of methods
in the standard library behave.</p>
<h2 id="using-bang-methods-safely">Using bang methods safely</h2>
<p>In the code that I was changing,
we had an intermediate array
that wasn’t being used.
I thought it was safe to mutate it
to avoid allocating more objects.
If you need to do something like this,
you can use <code class="language-plaintext highlighter-rouge">reject!</code> inside <code class="language-plaintext highlighter-rouge">tap</code> like this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="mi">1</span><span class="p">,</span><span class="mi">3</span><span class="p">,</span><span class="mi">5</span><span class="p">].</span><span class="nf">tap</span> <span class="p">{</span> <span class="o">|</span><span class="n">n</span><span class="o">|</span> <span class="n">n</span><span class="p">.</span><span class="nf">reject!</span><span class="p">(</span><span class="o">&</span><span class="ss">:even?</span><span class="p">)</span> <span class="p">}</span> <span class="c1">#=> [1, 3, 5]</span>
</code></pre></div></div>
<p>There’s also another alternative
in case of <code class="language-plaintext highlighter-rouge">reject!</code> -
<code class="language-plaintext highlighter-rouge">Array#delete_if</code> works like
how I expected <code class="language-plaintext highlighter-rouge">reject!</code> to work
and returns the array,
irrespective of whether it was mutated or not.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">].</span><span class="nf">delete_if</span><span class="p">(</span><span class="o">&</span><span class="ss">:even?</span><span class="p">)</span> <span class="c1">#=> [1, 3]</span>
<span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">].</span><span class="nf">delete_if</span><span class="p">(</span><span class="o">&</span><span class="ss">:even?</span><span class="p">)</span> <span class="c1">#=> [1, 3, 5]</span>
</code></pre></div></div>
<p>In the end, I decided not to change the code.
I find <code class="language-plaintext highlighter-rouge">reject</code> to be the most readable option,
and profiling the code
didn’t show any significant gains
from avoiding the allocations.</p>
<h2 id="so-what-does--indicate">So what does <code class="language-plaintext highlighter-rouge">!</code> indicate?</h2>
<p>Matz, the creator of Ruby,
has this to say
about bang methods:</p>
<blockquote>
<p>The bang (!) does not mean “destructive”
nor lack of it mean non destructive either.
The bang sign means
“the bang version is more dangerous
than its non bang counterpart; handle with care”.</p>
</blockquote>
<p>This is what Ruby docs say:</p>
<blockquote>
<p>…by convention,
a method with an exclamation point or bang is considered dangerous.
In Ruby’s core library the dangerous method implies that
when a method ends with a bang (!),
it indicates that unlike its non-bang equivalent, permanently modifies its receiver.</p>
</blockquote>
<p>Typically,
in the Ruby standard library,
this means that a bang method
returns either the mutated object,
or <code class="language-plaintext highlighter-rouge">nil</code> if there’s nothing to mutate.</p>
<p>However,
not all dangerous methods
are bang methods.
There are <code class="language-plaintext highlighter-rouge">Array</code> methods like
<code class="language-plaintext highlighter-rouge">push</code>, <code class="language-plaintext highlighter-rouge">pop</code>, <code class="language-plaintext highlighter-rouge">prepend</code>, etc
which will mutate the object,
but they don’t have the <code class="language-plaintext highlighter-rouge">!</code> suffix
because they don’t have
a safe counterpart.</p>In Ruby, method names can be suffixed with !. These are often called “bang” methods. This is used to indicate that it is a dangerous counterpart of the non-bang version of the method.Highlighting liquid code in Jekyll2023-08-02T00:00:00+00:002023-08-02T00:00:00+00:00https://nithinbekal.com/posts/jekyll-liquid-highlight<p>Recently I discovered that
any liquid code we put in a Jekyll post
gets evaluated by the liquid engine.
This means that if you put in
<a href="/posts/jekyll-posts-by-year/">a code snippet for listing posts</a>,
you will see the actual HTML rendered,
with all the posts.</p>
<p>As an example,
try putting some liquid tags inside a markdown code block:</p>
<div class="language-liquid highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{{</span><span class="w"> </span><span class="s2">"foo"</span><span class="w"> </span><span class="p">|</span><span class="w"> </span><span class="nf">upcase</span><span class="w"> </span><span class="p">}}</span>
</code></pre></div></div>
<p>This is going to look like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>FOO
</code></pre></div></div>
<p>So how are we seeing the raw liquid code above?
There are two ways
we can prevent liquid code from being evaluated.</p>
<p>The first option is to
<strong>wrap the code in the
<code>{% raw %}</code> tag</strong>.
This would look like this:</p>
<div class="language-liquid highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{%</span><span class="w"> </span><span class="nt">raw</span><span class="w"> </span><span class="p">%}</span> {{ "foo" | upcase }} <span class="p">{%</span><span class="w"> </span><span class="nt">endraw</span><span class="w"> </span><span class="p">%}</span>
</code></pre></div></div>
<p>The second option is to <strong>use HTML entities</strong>
like <code class="language-plaintext highlighter-rouge">&#123;</code> and <code class="language-plaintext highlighter-rouge">&#125;</code>
for the opening and closing curly braces, respectively.
We would need to type something like this
in the markdown file:</p>
<div class="language-liquid highlighter-rouge"><div class="highlight"><pre class="highlight"><code>&#123;&#123; "foo" | upcase &#125;&#125;
</code></pre></div></div>
<p>Unfortunately,
if we put this
inside markdown fenced code blocks
using <code>```</code>,
the actual entity will be rendered as-is
rather than the intended character.
We will need to explicitly wrap the code
with
<code><pre></code>
and
<code><code></code>
tags
to make this work.</p>
<h2 id="meta-problem-how-do-we-print-the-rawendraw-tags">Meta problem: How do we print the raw/endraw tags?</h2>
<p>At this point,
you might be wondering:
If the raw/endraw tags
are used to escape the other tags,
how is the <code>{% endraw %}</code> tag
appearing on this page?
Because if you tried wrapping
<code>{% endraw %}</code>
itself inside another set of raw/endraw tags,
it will look like this:</p>
<div class="language-liquid highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{%</span><span class="w"> </span><span class="nt">raw</span><span class="w"> </span><span class="p">%}{%</span><span class="w"> </span><span class="nt">endraw</span><span class="w"> </span><span class="p">%}{%</span><span class="w"> </span><span class="nt">endraw</span><span class="w"> </span><span class="p">%}</span>
</code></pre></div></div>
<p>The first endraw tag
will terminate the block,
and we will end up with a liquid error!</p>
<p>To solve this problem,
we will need to break up these tags
so that the opening tag token “<code class="language-plaintext highlighter-rouge">{%</code>”
is a separate liquid variable.
You might want to put this line
before the code block,
or it will appear as an empty line.</p>
<div class="language-liquid highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{%</span><span class="w"> </span><span class="nt">assign</span><span class="w"> </span><span class="nv">open</span><span class="w"> </span><span class="o">=</span><span class="w"> </span><span class="s2">"{%"</span><span class="w"> </span><span class="p">%}</span>
</code></pre></div></div>
<p>Now to render
the <code>{% raw %}</code> tag,
we can do something like this:</p>
<div class="language-liquid highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{{</span><span class="w"> </span><span class="nv">open</span><span class="w"> </span><span class="p">}}</span> raw %}
</code></pre></div></div>
<p>We can now go back to
printing the code
with raw and endraw tags.
To display the string
“<code>{% raw %}{% endraw %}{% endraw %}</code>”,
we need to write:</p>
<div class="language-liquid highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{{</span><span class="w"> </span><span class="nv">open</span><span class="w"> </span><span class="p">}}</span> raw %}<span class="p">{{</span><span class="w"> </span><span class="nv">open</span><span class="w"> </span><span class="p">}}</span> endraw %}<span class="p">{{</span><span class="w"> </span><span class="nv">open</span><span class="w"> </span><span class="p">}}</span> endraw %}
</code></pre></div></div>
<p>In case you are wondering,
what kind of liquid code
did it take to print
the code block above,
here you are:</p>
<div class="language-liquid highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{%</span><span class="w"> </span><span class="nt">raw</span><span class="w"> </span><span class="p">%}</span>{{ open }} raw %}{{ open }} endraw %}{{ open }} endraw %}<span class="p">{%</span><span class="w"> </span><span class="nt">endraw</span><span class="w"> </span><span class="p">%}</span>
</code></pre></div></div>
<p>Now, we really have to stop.
If you’re curious
what it took to print this line,
you can always look up
<a href="https://github.com/nithinbekal/nithinbekal.github.io/blob/7f85a0c4a8fe1597c6bc08e1cb49b7bde46461c1/_posts/2023-08-02-jekyll-liquid-highlight.md">the markdown code for this post</a>.</p>
<p>Here are a couple of other articles
that helped me figure out
how to escape liquid tags in Jekyll.
(Bonus: you can look at their markdown source
to see it in action.)</p>
<ul>
<li><a href="https://blog.slaks.net/2013-06-10/jekyll-endraw-in-code/">Writing the endraw tag in Jekyll code blocks</a>
(<a href="https://raw.githubusercontent.com/SLaks/SLaks.Blog/gh-pages/_posts/2013-06-10-jekyll-endraw-in-code.md">markdown</a>)</li>
<li><a href="https://rmaicle.github.io/posts/71xXldY8KeKXWgw">Liquid ‘raw’ Tag and Fenced Code Blocks</a>
(<a href="https://github.com/rmaicle/rmaicle.github.io/blob/master/_posts/jekyll/2014-11-12-Liquid%20'raw'%20Tag%20and%20Fenced%20Code%20Blocks.md">markdown</a>)</li>
</ul>
<hr />
<p>PS. I couldn’t help it.
Here’s the liquid code for that last code block.
This time, I really am done! ;)</p>
<div class="language-liquid highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{{</span><span class="w"> </span><span class="nv">open</span><span class="w"> </span><span class="p">}}</span> raw %}<span class="p">{%</span><span class="w"> </span><span class="nt">raw</span><span class="w"> </span><span class="p">%}</span>{{ open }} raw %}{{ open }} endraw %}{{ open }} endraw %}<span class="p">{%</span><span class="w"> </span><span class="nt">endraw</span><span class="w"> </span><span class="p">%}{{</span><span class="w"> </span><span class="nv">open</span><span class="w"> </span><span class="p">}}</span> endraw %}
</code></pre></div></div>Recently I discovered that any liquid code we put in a Jekyll post gets evaluated by the liquid engine. This means that if you put in a code snippet for listing posts, you will see the actual HTML rendered, with all the posts.Ruby and Jupyter Notebooks2023-08-01T00:00:00+00:002023-08-01T00:00:00+00:00https://nithinbekal.com/posts/ruby-jupyter-notebooks<p>Today, I was working through
Andrew Ng’s excellent short course
on <a href="https://www.deeplearning.ai/short-courses/">building systems with ChatGPT</a>.
The course uses Jupyter notebooks
for running the Python code.</p>
<p>While figuring out Python syntax,
one question came to mind:
does Ruby have something similar to Jupyter notebooks?
Turns out that you can just use Jupyter notebooks with Ruby
if you setup the <a href="https://github.com/SciRuby/iruby">iruby kernel</a>.
This kernel is available as a gem,
and the setup is really simple.</p>
<h2 id="setting-up-jupyter-and-iruby">Setting up Jupyter and iruby</h2>
<p>First, install Jupyter using Python’s pip installer.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>pip <span class="nb">install </span>jupyter
</code></pre></div></div>
<p>Next, install the iruby gem.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gem <span class="nb">install </span>iruby
</code></pre></div></div>
<p>Finally, register iruby as a kernel for Jupyter.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>iruby register <span class="nt">--force</span>
</code></pre></div></div>
<p>Open the jupyter notebook:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>jupyter notebook
</code></pre></div></div>
<p>Now if you go Kernel -> Change kernel
in the toolbar,
you will see Ruby as one of the options.</p>
<p>At this point, we’re ready to create our first Ruby notebook.
When you create a new notebook,
you will see Ruby as one of the available kernels.
You can pick that and run <code class="language-plaintext highlighter-rouge">puts "Hello, world!"</code>
just to make sure you have it working correctly.</p>
<h2 id="creating-our-first-notebook">Creating our first notebook</h2>
<p>I wanted to play around with the OpenAI API,
so I need to install the
<a href="https://github.com/alexrudall/ruby-openai"><code class="language-plaintext highlighter-rouge">ruby-openai</code> gem</a>.
This can be done in the notebook via <a href="/posts/bundler-inline-gemfile/">bundler’s inline gemfile definition</a>.
Put this in the first cell:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s2">"bundler/inline"</span>
<span class="n">gemfile</span> <span class="k">do</span>
<span class="n">source</span> <span class="s2">"https://rubygems.org"</span>
<span class="n">gem</span> <span class="s2">"dotenv"</span><span class="p">,</span> <span class="ss">require: </span><span class="s2">"dotenv/load"</span>
<span class="n">gem</span> <span class="s2">"ruby-openai"</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Aside from <code class="language-plaintext highlighter-rouge">ruby-openai</code>,
I have also included <code class="language-plaintext highlighter-rouge">dotenv</code> here
so that I can store OpenAI credentials
in a <code class="language-plaintext highlighter-rouge">.env</code> file in the notebooks folder.
This <code class="language-plaintext highlighter-rouge">.env</code> file should look like:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>OPENAI_ACCESS_TOKEN=your_key
</code></pre></div></div>
<p>The <code class="language-plaintext highlighter-rouge">require: "dotenv/load"</code> in the gemfile definition
reads the <code class="language-plaintext highlighter-rouge">.env</code> file and updates the environment with it.
We can now use it
to initialize the OpenAI client:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">client</span> <span class="o">=</span> <span class="no">OpenAI</span><span class="o">::</span><span class="no">Client</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">access_token: </span><span class="no">ENV</span><span class="p">[</span><span class="s2">"OPENAI_ACCESS_TOKEN"</span><span class="p">])</span>
</code></pre></div></div>
<p>And use one of the examples from the <code class="language-plaintext highlighter-rouge">ruby-openai</code> README:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">response</span> <span class="o">=</span> <span class="n">client</span><span class="p">.</span><span class="nf">chat</span><span class="p">(</span>
<span class="ss">parameters: </span><span class="p">{</span>
<span class="ss">model: </span><span class="s2">"gpt-3.5-turbo"</span><span class="p">,</span>
<span class="ss">messages: </span><span class="p">[{</span> <span class="ss">role: </span><span class="s2">"user"</span><span class="p">,</span> <span class="ss">content: </span><span class="s2">"Hello!"</span><span class="p">}],</span>
<span class="ss">temperature: </span><span class="mf">0.7</span><span class="p">,</span>
<span class="p">}</span>
<span class="p">)</span>
</code></pre></div></div>
<p>If you have a valid OpenAI access token,
you should see a response
containing something like
“Hello! How can I assist you today?”.</p>
<p>And that’s it!
That’s all it took
to get to a working playground
for exploring ChatGPT.
I’m excited to see
how I can make notebooks
a part of my workflow.
I can see notebooks being useful
when building up small scripts,
or when learning about
how a feature of Ruby works.
I’m also considering
using a notebook
to draft this year’s
<a href="/posts/ruby-3-2/">what’s new in Ruby</a> article
using markdown cells mixed with code blocks.</p>Today, I was working through Andrew Ng’s excellent short course on building systems with ChatGPT. The course uses Jupyter notebooks for running the Python code.Jekyll: Skip analytics scripts in development2023-07-27T00:00:00+00:002023-07-27T00:00:00+00:00https://nithinbekal.com/posts/jekyll-environment<p>When <a href="/posts/dark-mode/">updating this site</a>,
I was looking for a way
to skip rendering some scripts locally.
For example, you might not want
Google Analytics to record
every local page visit
you make when proofreading a post,
or hide the Disqus comments section.</p>
<p>Jekyll exposes the <a href="https://jekyllrb.com/docs/configuration/environments/">environment</a>
that it is running in
via <code class="language-plaintext highlighter-rouge">jekyll.environment</code>.
This means that you can skip these scripts locally
by doing something like this:</p>
<div class="language-liquid highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{%</span><span class="w"> </span><span class="kr">if</span><span class="w"> </span><span class="nv">jekyll</span><span class="p">.</span><span class="nv">environment</span><span class="w"> </span><span class="o">==</span><span class="w"> </span><span class="s2">"production"</span><span class="w"> </span><span class="p">%}</span>
<!-- script goes here -->
<span class="p">{%</span><span class="w"> </span><span class="kr">endif</span><span class="w"> </span><span class="p">%}</span>
</code></pre></div></div>
<p>If you’re using Github Pages,
the environment is already set as <code class="language-plaintext highlighter-rouge">production</code> for you.
However, if you are compiling it for production yourself,
you can set the <code class="language-plaintext highlighter-rouge">JEKYLL_ENV</code> environment variable
when running <code class="language-plaintext highlighter-rouge">jekyll build</code>.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nv">JEKYLL_ENV</span><span class="o">=</span>production jekyll build
</code></pre></div></div>When updating this site, I was looking for a way to skip rendering some scripts locally. For example, you might not want Google Analytics to record every local page visit you make when proofreading a post, or hide the Disqus comments section.