Jekyll2021-11-02T09:32:17+10:00http://simon.rentzke.com/feed.xmlSimon RentzkeExperienced in systems architecture, scaling large rails web applications, devops and managing remote lean teams.Simon Rentzkesimon@rentzke.comCustom local settings for your Rails App2021-09-02T17:16:00+10:002021-09-02T17:16:00+10:00http://simon.rentzke.com/custom-local-settings-for-your-rails-app<p>Sometimes, especially in bigger teams, there maybe some dev related settings that you’d prefer not to turn on and you do not want to check-in your changes into GitHub.</p>
<p>For example turning off Bullet, or adding a custom Rspec setting. All you’ll need to do is, add this to .gitignore:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code># .gitignore
/config/initializers/zzz_custom*.rb
</code></pre></div></div>
<p>Then create a file in your initializers directory with your patch:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># config/initializers/zzz_custom_rspec.rb</span>
<span class="k">if</span> <span class="no">Rails</span><span class="p">.</span><span class="nf">env</span><span class="p">.</span><span class="nf">test?</span>
<span class="c1"># this is for running the only--failures task</span>
<span class="no">RSpec</span><span class="p">.</span><span class="nf">configure</span> <span class="k">do</span> <span class="o">|</span><span class="n">c</span><span class="o">|</span>
<span class="n">c</span><span class="p">.</span><span class="nf">example_status_persistence_file_path</span> <span class="o">=</span> <span class="s2">"tmp/rspec_examples.txt"</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Or another one:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># config/initializers/zzz_custom_bullet.rb</span>
<span class="k">if</span> <span class="no">Rails</span><span class="p">.</span><span class="nf">env</span><span class="p">.</span><span class="nf">development?</span>
<span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">after_initialize</span> <span class="k">do</span>
<span class="no">Bullet</span><span class="p">.</span><span class="nf">rails_logger</span> <span class="o">=</span> <span class="kp">false</span>
<span class="no">Bullet</span><span class="p">.</span><span class="nf">enable</span> <span class="o">=</span> <span class="kp">false</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Then you’re good to go, without stepping on your fellow teammates toes.</p>Simon Rentzkesimon@rentzke.comSometimes, especially in bigger teams, there maybe some dev related settings that you’d prefer not to turn on and you do not want to check-in your changes into GitHub.Force Rails to drop Postgres DB even when there are other session using the database2021-09-02T17:04:00+10:002021-09-02T17:04:00+10:00http://simon.rentzke.com/force-rails-to-db-drop-postgres-db-when-there-are-other-session-using-the-database<p>When using multiple branches that have slightly different db schemas - I have found myself needing to recreate my local development and test db fairly often.</p>
<p>For example when running <code class="language-plaintext highlighter-rouge">rails db:reset</code> often I get an error message like this:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>PG::ObjectInUse: ERROR: database "app_test" is being accessed by other users
DETAIL: There is 1 other session using the database.
Couldn't drop database 'app_test'
rails aborted!
ActiveRecord::StatementInvalid: PG::ObjectInUse: ERROR: database "app_test" is being accessed by other users
DETAIL: There is 1 other session using the database.
Caused by:
PG::ObjectInUse: ERROR: database "app_test" is being accessed by other users
DETAIL: There is 1 other session using the database.
</code></pre></div></div>
<p>Which is easy enough to fix (by killing the postgres sessions) - but I’d prefer not to have to manually do that ever, as I need not care about any sessions that are attached.</p>
<p>There is a super simple monkey patch that I’ve added to ensure that I can always kill the db:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="c1"># config/initializers/monkey_patch_activerecord.rb</span>
<span class="k">class</span> <span class="nc">ActiveRecord::Tasks::PostgreSQLDatabaseTasks</span>
<span class="k">def</span> <span class="nf">drop</span>
<span class="n">establish_master_connection</span>
<span class="n">connection</span><span class="p">.</span><span class="nf">execute</span> <span class="s2">"DROP DATABASE IF EXISTS </span><span class="se">\"</span><span class="si">#{</span><span class="n">db_config</span><span class="p">.</span><span class="nf">database</span><span class="si">}</span><span class="se">\"</span><span class="s2"> WITH (FORCE)"</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>The important bit above, is the <code class="language-plaintext highlighter-rouge">WITH (FORCE)</code> – of course :)</p>Simon Rentzkesimon@rentzke.comWhen using multiple branches that have slightly different db schemas - I have found myself needing to recreate my local development and test db fairly often.Moving from activerecord session storage to redis2021-02-23T17:35:00+10:002021-02-23T17:35:00+10:00http://simon.rentzke.com/moving-from-activerecord-session-storage-to-redis<p>It’s not a great idea to have your sessions stored in the database, as it can cause all sorts of unnecessary load problems that can be offset to another service.</p>
<p>The easiest and most performant option is to use a <a href="https://api.rubyonrails.org/classes/ActionDispatch/Session/CookieStore.html">cookie store</a>, but if you’re keen on migrating existing sessions and still having control of your sessions, I’d recommend moving to Redis.<</p>
<p>It’s quite straightforward to make the move. All you need to do is install the Redis Actionpack gem, <code class="language-plaintext highlighter-rouge">gem 'redis-actionpack'</code>.</p>
<p>Then set your session store config, to point to your Redis service.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="no">Rails</span><span class="p">.</span><span class="nf">application</span><span class="p">.</span><span class="nf">config</span><span class="p">.</span><span class="nf">session_store</span> <span class="ss">:redis_store</span><span class="p">,</span>
<span class="ss">servers: </span><span class="p">[</span><span class="no">REDIS_SESSIONS_URL</span><span class="p">],</span>
<span class="ss">key: </span><span class="s1">'_session_key'</span><span class="p">,</span>
<span class="ss">httponly: </span><span class="kp">true</span><span class="p">,</span>
<span class="ss">secure: </span><span class="kp">true</span>
</code></pre></div></div>
<p>Then migrate all of your sessions over from ActiveRecord onto Redis, so that your users are not logged out:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="n">redis</span> <span class="o">=</span> <span class="no">Redis</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">url: </span><span class="no">REDIS_SESSIONS_URL</span><span class="p">)</span>
<span class="n">old_sessions</span> <span class="o">=</span> <span class="no">ActiveRecord</span><span class="o">::</span><span class="no">SessionStore</span><span class="o">::</span><span class="no">Session</span>
<span class="n">old_sessions</span><span class="p">.</span><span class="nf">find_each</span> <span class="k">do</span> <span class="o">|</span><span class="n">session</span><span class="o">|</span>
<span class="c1"># create each session into Redis, dumping the object appropriately with Marshal</span>
<span class="n">redis</span><span class="p">.</span><span class="nf">setex</span> <span class="no">Rack</span><span class="o">::</span><span class="no">Session</span><span class="o">::</span><span class="no">SessionId</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="n">session</span><span class="p">.</span><span class="nf">session_id</span><span class="p">).</span><span class="nf">private_id</span><span class="p">,</span>
<span class="mi">1</span><span class="p">.</span><span class="nf">month</span><span class="p">.</span><span class="nf">to_i</span><span class="p">,</span>
<span class="no">Marshal</span><span class="p">.</span><span class="nf">dump</span><span class="p">(</span><span class="n">session</span><span class="p">.</span><span class="nf">data</span><span class="p">).</span><span class="nf">to_s</span><span class="p">.</span><span class="nf">force_encoding</span><span class="p">(</span><span class="no">Encoding</span><span class="o">::</span><span class="no">BINARY</span><span class="p">)</span>
<span class="k">end</span>
</code></pre></div></div>Simon Rentzkesimon@rentzke.comIt’s not a great idea to have your sessions stored in the database, as it can cause all sorts of unnecessary load problems that can be offset to another service.Using Google Cloud Functions to receive, manipulate, and send the data to another web service2020-11-27T18:22:00+10:002020-11-27T18:22:00+10:00http://simon.rentzke.com/using-google-cloud-functions-to-quickly-manipulate-data-from-two-different-web-services<p>I recently had an issue that <a href="https://coveralls.io/">Coveralls</a> did not support the output from the <a href="https://www.cloudbees.com/products/codeship">Codeship</a> build webhook. So I needed to send data from the Codeship webhook to an intermediary, change the format of the json, and then send it to Coveralls.</p>
<p>There are a lot of services that can do similar things, like <a href="https://zapier.com/">Zapier</a> or <a href="https://ifttt.com/">IFTT</a>. But what I needed was incredibly simple, and a Google Cloud function, with a few lines of code suits this use case quite well.</p>
<p>The payload from the webhook from <a href="https://apidocs.codeship.com/v2/builds/get-build">Codeship</a> generally looks like this:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"build"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"uuid"</span><span class="p">:</span><span class="w"> </span><span class="s2">"20b4a690-6a03-0145-d6ec-0000000000"</span><span class="p">,</span><span class="w">
</span><span class="nl">"project_uuid"</span><span class="p">:</span><span class="w"> </span><span class="s2">"7b3596c0-560e-0135-5b18-000000000000"</span><span class="p">,</span><span class="w">
</span><span class="nl">"organization_uuid"</span><span class="p">:</span><span class="w"> </span><span class="s2">"721cea10-b695-0134-5b94-000000000000"</span><span class="p">,</span><span class="w">
</span><span class="nl">"ref"</span><span class="p">:</span><span class="w"> </span><span class="s2">"heads/master"</span><span class="p">,</span><span class="w">
</span><span class="nl">"commit_sha"</span><span class="p">:</span><span class="w"> </span><span class="s2">"575a1521fff1466c3ed9fae9d390e0ffffffffff"</span><span class="p">,</span><span class="w">
</span><span class="nl">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"success"</span><span class="p">,</span><span class="w">
</span><span class="nl">"username"</span><span class="p">:</span><span class="w"> </span><span class="s2">"dennisnewel"</span><span class="p">,</span><span class="w">
</span><span class="nl">"commit_message"</span><span class="p">:</span><span class="w"> </span><span class="s2">"Update to API documentation"</span><span class="p">,</span><span class="w">
</span><span class="nl">"finished_at"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2017-08-23T07:26:01.877Z"</span><span class="p">,</span><span class="w">
</span><span class="nl">"allocated_at"</span><span class="p">:</span><span class="w"> </span><span class="kc">null</span><span class="p">,</span><span class="w">
</span><span class="nl">"queued_at"</span><span class="p">:</span><span class="w"> </span><span class="s2">"2017-08-23T07:24:41.327Z"</span><span class="p">,</span><span class="w">
</span><span class="nl">"links"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"pipelines"</span><span class="p">:</span><span class="w"> </span><span class="s2">"https://api.codeship.com/v2/organizations/721cea10-b695-0134-5b94-000000000000/projects/7b3596c0-560e-0135-5b18-000000000000/builds/20b4a690-6a03-0135-d6ec-000000000000/pipelines"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>And the payload that <a href="https://docs.coveralls.io/parallel-build-webhook">Coveralls</a> needs looks something like this:</p>
<div class="language-json highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">{</span><span class="w">
</span><span class="nl">"payload"</span><span class="p">:</span><span class="w"> </span><span class="p">{</span><span class="w">
</span><span class="nl">"build_num"</span><span class="p">:</span><span class="w"> </span><span class="mi">1234</span><span class="p">,</span><span class="w">
</span><span class="nl">"status"</span><span class="p">:</span><span class="w"> </span><span class="s2">"done"</span><span class="w">
</span><span class="p">}</span><span class="w">
</span><span class="p">}</span><span class="w">
</span></code></pre></div></div>
<p>First I setup environment variables for the <code class="language-plaintext highlighter-rouge">REPO_TOKEN</code>, and then these few lines of js are all you need to send data from the webhook.</p>
<div class="language-js highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="kd">const</span> <span class="nx">fetch</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="dl">'</span><span class="s1">node-fetch</span><span class="dl">'</span><span class="p">);</span>
<span class="nx">exports</span><span class="p">.</span><span class="nx">coveralls</span> <span class="o">=</span> <span class="k">async</span> <span class="p">(</span><span class="nx">req</span><span class="p">,</span> <span class="nx">res</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="kd">let</span> <span class="nx">data</span><span class="p">;</span>
<span class="nx">data</span> <span class="o">=</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">parse</span><span class="p">(</span><span class="nx">req</span><span class="p">.</span><span class="nx">rawBody</span><span class="p">);</span>
<span class="c1">// console.log(data.build.build_id);</span>
<span class="c1">// console.log(data.build.status);</span>
<span class="k">if</span> <span class="p">(</span><span class="nx">data</span><span class="p">.</span><span class="nx">build</span><span class="p">.</span><span class="nx">status</span> <span class="o">==</span> <span class="dl">'</span><span class="s1">success</span><span class="dl">'</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">coverallsResponse</span> <span class="o">=</span>
<span class="k">await</span> <span class="nx">fetch</span><span class="p">(</span><span class="s2">`https://coveralls.io/webhook.json?repo_token=</span><span class="p">${</span><span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">REPO_TOKEN</span><span class="p">}</span><span class="s2">`</span><span class="p">,</span>
<span class="p">{</span>
<span class="na">method</span><span class="p">:</span> <span class="dl">'</span><span class="s1">POST</span><span class="dl">'</span><span class="p">,</span>
<span class="na">body</span><span class="p">:</span> <span class="nx">JSON</span><span class="p">.</span><span class="nx">stringify</span><span class="p">({</span>
<span class="na">payload</span><span class="p">:</span> <span class="p">{</span>
<span class="na">build_num</span><span class="p">:</span> <span class="nx">data</span><span class="p">.</span><span class="nx">build</span><span class="p">.</span><span class="nx">build_id</span><span class="p">,</span>
<span class="na">status</span><span class="p">:</span> <span class="dl">'</span><span class="s1">done</span><span class="dl">'</span>
<span class="p">}</span>
<span class="p">}),</span>
<span class="na">headers</span><span class="p">:</span> <span class="p">{</span>
<span class="na">Accept</span><span class="p">:</span> <span class="dl">'</span><span class="s1">application/json</span><span class="dl">'</span><span class="p">,</span>
<span class="dl">'</span><span class="s1">Content-Type</span><span class="dl">'</span><span class="p">:</span> <span class="dl">'</span><span class="s1">application/json</span><span class="dl">'</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">).</span><span class="nx">then</span><span class="p">((</span><span class="nx">response</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="k">if</span> <span class="p">(</span><span class="o">!</span><span class="nx">response</span><span class="p">.</span><span class="nx">ok</span><span class="p">)</span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="s2">`Error: Response from Coveralls was </span><span class="p">${</span><span class="nx">response</span><span class="p">.</span><span class="nx">statusText</span><span class="p">}</span><span class="s2">`</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}).</span><span class="k">catch</span><span class="p">((</span><span class="nx">error</span><span class="p">)</span> <span class="o">=></span> <span class="p">{</span>
<span class="nx">console</span><span class="p">.</span><span class="nx">error</span><span class="p">(</span><span class="nx">error</span><span class="p">.</span><span class="nx">message</span><span class="p">);</span>
<span class="p">})</span>
<span class="p">}</span>
<span class="nx">res</span><span class="p">.</span><span class="nx">status</span><span class="p">(</span><span class="mi">200</span><span class="p">).</span><span class="nx">send</span><span class="p">(</span><span class="s2">`Done.`</span><span class="p">);</span>
<span class="p">}</span>
</code></pre></div></div>Simon Rentzkesimon@rentzke.comI recently had an issue that Coveralls did not support the output from the Codeship build webhook. So I needed to send data from the Codeship webhook to an intermediary, change the format of the json, and then send it to Coveralls.Dump csv reports using Ruby on Rails and Postgres2018-08-31T14:57:50+10:002018-08-31T14:57:50+10:00http://simon.rentzke.com/dump-csv-reports-using-ruby-on-rails-and-postgres<p>This method is the most efficient, super simple way to generate CSV files with Rails. It uses an active record connection and the <a href="https://www.postgresql.org/docs/10/static/sql-copy.html">Postgres COPY command</a> writes each csv generated line from Postgres to a file.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="no">File</span><span class="p">.</span><span class="nf">open</span><span class="p">(</span><span class="n">file_path</span><span class="p">,</span> <span class="s1">'w'</span><span class="p">)</span> <span class="k">do</span> <span class="o">|</span><span class="n">f</span><span class="o">|</span>
<span class="no">ActiveRecord</span><span class="o">::</span><span class="no">Base</span><span class="p">.</span><span class="nf">connection</span><span class="p">.</span><span class="nf">raw_connection</span><span class="p">.</span><span class="nf">copy_data</span> <span class="s2">"COPY (SELECT * FROM users) TO STDOUT WITH (FORMAT CSV, HEADER TRUE, FORCE_QUOTE *);"</span> <span class="k">do</span>
<span class="k">while</span> <span class="n">line</span> <span class="o">=</span> <span class="n">connection</span><span class="p">.</span><span class="nf">raw_connection</span><span class="p">.</span><span class="nf">get_copy_data</span> <span class="k">do</span>
<span class="n">f</span><span class="p">.</span><span class="nf">write</span> <span class="n">line</span><span class="p">.</span><span class="nf">force_encoding</span><span class="p">(</span><span class="s1">'UTF-8'</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Leveraging Postgres to do this, results in tens of millions of rows being able to be exported in under 5 mins.</p>Simon Rentzkesimon@rentzke.comThis method is the most efficient, super simple way to generate CSV files with Rails. It uses an active record connection and the Postgres COPY command writes each csv generated line from Postgres to a file.Support SVG’s and PNG/JPG in your Rails Dragonfly workflow2017-07-18T13:20:51+10:002017-07-18T13:20:51+10:00http://simon.rentzke.com/support-svgs-and-png-slash-jpg-in-your-rails-dragonfly-workflow<p>I was struggling to find a solution to easily provide thumb pngs, as well as display svgs inline with the <em>img</em> tag with the ruby gem Dragonfly.</p>
<p>For example something simple like this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code>
<span class="o">=</span> <span class="n">image_tag</span> <span class="n">user</span><span class="p">.</span><span class="nf">image</span><span class="p">.</span><span class="nf">thumb</span><span class="p">(</span><span class="s1">'100x100#'</span><span class="p">).</span><span class="nf">url</span>
</code></pre></div></div>
<p>But that breaks when showing SVG files, as it looks like it strips out a bunch of important info from the file.</p>
<p>To set the dimensions of the svg correctly. You can add an additional gem, <em>gem ‘dragonfly_svg’</em> and you’ll also need to add that as a plugin to your <em>dragonfly.rb</em> file.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">require</span> <span class="s1">'dragonfly'</span>
<span class="c1"># Configure</span>
<span class="no">Dragonfly</span><span class="p">.</span><span class="nf">app</span><span class="p">.</span><span class="nf">configure</span> <span class="k">do</span>
<span class="n">plugin</span> <span class="ss">:imagemagick</span>
<span class="n">plugin</span> <span class="ss">:svg</span>
<span class="o">...</span>
</code></pre></div></div>
<p>To render the SVG correctly you’d write this:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">=</span> <span class="n">image_tag</span> <span class="n">user</span><span class="p">.</span><span class="nf">image</span><span class="p">.</span><span class="nf">set_dimensions</span><span class="p">(</span><span class="mi">100</span><span class="p">,</span><span class="mi">100</span><span class="p">).</span><span class="nf">url</span>
</code></pre></div></div>
<p>So, in order to support both seamlessly. I built a custom Dragonfly processor.</p>
<p>Then you can setup a custom processor, that checks the mime type of the image, and processes it either way.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="n">processor</span> <span class="ss">:custom_thumb</span> <span class="k">do</span> <span class="o">|</span><span class="n">content</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="o">|</span>
<span class="k">if</span> <span class="n">content</span><span class="p">.</span><span class="nf">mime_type</span> <span class="o">==</span> <span class="s1">'image/svg+xml'</span>
<span class="n">dimensions</span> <span class="o">=</span> <span class="n">args</span><span class="p">.</span><span class="nf">first</span><span class="p">.</span><span class="nf">split</span><span class="p">(</span><span class="s1">'x'</span><span class="p">)</span>
<span class="n">content</span><span class="p">.</span><span class="nf">process!</span><span class="p">(</span><span class="ss">:set_dimensions</span><span class="p">,</span> <span class="n">dimensions</span><span class="p">[</span><span class="mi">0</span><span class="p">].</span><span class="nf">to_i</span><span class="p">,</span> <span class="n">dimensions</span><span class="p">[</span><span class="mi">1</span><span class="p">].</span><span class="nf">to_i</span><span class="p">)</span>
<span class="k">else</span>
<span class="n">content</span><span class="p">.</span><span class="nf">process!</span><span class="p">(</span><span class="ss">:thumb</span><span class="p">,</span> <span class="o">*</span><span class="n">args</span><span class="p">)</span>
<span class="k">end</span>
<span class="k">end</span>
</code></pre></div></div>
<p>Then, throughout the code, you can easily refer to the <em>custom_thumb</em> processor:</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="o">=</span> <span class="n">image_tag</span> <span class="n">user</span><span class="p">.</span><span class="nf">image</span><span class="p">.</span><span class="nf">custom_thumb</span><span class="p">(</span><span class="s1">'100x100#'</span><span class="p">).</span><span class="nf">url</span>
</code></pre></div></div>Simon Rentzkesimon@rentzke.comI was struggling to find a solution to easily provide thumb pngs, as well as display svgs inline with the img tag with the ruby gem Dragonfly.Removing sensitive data from GitHub wiki revision history2016-10-12T17:48:42+10:002016-10-12T17:48:42+10:00http://simon.rentzke.com/removing-sensitive-data-from-github-wiki-revision-history<p>If you, like me, by mistake added a password to a Github Wiki, and would like to remove it from history. It’s actually quite easy.</p>
<p>The whole wiki, is actually a git repo, which you can checkout locally like this:</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>git clone https://github.com/<user>/<repo>.wiki.git
</code></pre></div></div>
<p>You can then do a rebase, to amend the the wiki changes (commits). Remove the sensitive commits, and then do a force push.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code> git rebase <span class="nt">--interactive</span> HEAD~2
git push <span class="nt">-f</span>
</code></pre></div></div>
<p><em>Please note:</em> that even though you won’t be able to see it on the UI, the sensitive data will still exist in the git history, if you would like to be sure that nothing exists, you can destroy the whole wiki’s history, by following this <a href="https://gist.github.com/hacksalot/72517b9b1c145116e89e">gist</a>.</p>Simon Rentzkesimon@rentzke.comIf you, like me, by mistake added a password to a Github Wiki, and would like to remove it from history. It’s actually quite easy.Playing with Googles Cloud Vision API2015-12-08T16:45:25+10:002015-12-08T16:45:25+10:00http://simon.rentzke.com/google-cloud-vision-api-on-ruby<p>The age of intelligent api’s is here.</p>
<p>If you want to get a taste of what Google’s new <a href="https://cloud.google.com/vision/">“Cloud Vision API”</a> can do. Sign up here <a href="https://cloud.google.com/vision/">https://cloud.google.com/vision/</a>.</p>
<p>Once you’re accepted, get an api key from the <a href="https://console.developers.google.com">Google console</a></p>
<p>Then at your fingertips you have:</p>
<ul>
<li>Optical Character Recognition</li>
<li>Face detection, and sentiment analysis i.e Happy, Sad</li>
<li>Landmark detection</li>
</ul>
<p>I wrote a tiny little ruby script if you want to give a whirl. This specifically uses the landmark detection api.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <span class="nb">require</span> <span class="s1">'base64'</span>
<span class="nb">require</span> <span class="s1">'json'</span>
<span class="nb">require</span> <span class="s1">'faraday'</span>
<span class="n">filepath</span> <span class="o">=</span> <span class="s1">'/path/to/my/image.jpg'</span>
<span class="n">content</span> <span class="o">=</span> <span class="no">Base64</span><span class="p">.</span><span class="nf">encode64</span><span class="p">(</span><span class="no">File</span><span class="p">.</span><span class="nf">binread</span><span class="p">(</span><span class="n">filepath</span><span class="p">))</span>
<span class="n">conn</span> <span class="o">=</span> <span class="no">Faraday</span><span class="p">.</span><span class="nf">new</span><span class="p">(</span><span class="ss">:url</span> <span class="o">=></span> <span class="s1">'https://vision.googleapis.com'</span><span class="p">)</span>
<span class="n">data</span> <span class="o">=</span> <span class="p">{</span><span class="ss">requests: </span><span class="p">[{</span><span class="s1">'image'</span> <span class="o">=></span> <span class="p">{</span><span class="s1">'content'</span> <span class="o">=></span> <span class="n">content</span><span class="p">},</span> <span class="s1">'features'</span> <span class="o">=></span> <span class="p">[{</span><span class="s1">'type'</span> <span class="o">=></span> <span class="s1">'LANDMARK_DETECTION'</span><span class="p">,</span> <span class="s1">'maxResults'</span> <span class="o">=></span> <span class="mi">10</span><span class="p">}]}]};</span>
<span class="n">response</span> <span class="o">=</span> <span class="n">conn</span><span class="p">.</span><span class="nf">post</span> <span class="k">do</span> <span class="o">|</span><span class="n">req</span><span class="o">|</span>
<span class="n">req</span><span class="p">.</span><span class="nf">url</span> <span class="s1">'/v1alpha1/images:annotate?key=<your api key>'</span>
<span class="n">req</span><span class="p">.</span><span class="nf">headers</span><span class="p">[</span><span class="s1">'Content-Type'</span><span class="p">]</span> <span class="o">=</span> <span class="s1">'application/json'</span>
<span class="n">req</span><span class="p">.</span><span class="nf">body</span> <span class="o">=</span> <span class="n">data</span><span class="p">.</span><span class="nf">to_json</span>
<span class="k">end</span>
<span class="nb">puts</span> <span class="n">response</span>
</code></pre></div></div>
<p>So here is a picture I took in 2006, without any geolocation info attached in exif data.</p>
<p><img src="http://s3.amazonaws.com/simonrentzke/stream_images/images/18/original.JPG?1449557791" alt="tahoe" title="Tahoe" /></p>
<p>What does the api return?</p>
<p>It gives a confidence score, a description of the landmark and its rough latitude and longitude, and a bunch of other cool stuff I’m yet to understand.</p>
<div class="language-ruby highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="p">[</span>
<span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="p">{</span>
<span class="s2">"landmarkAnnotations"</span> <span class="o">=></span> <span class="p">[</span>
<span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="p">{</span>
<span class="s2">"mid"</span> <span class="o">=></span> <span class="s2">"/m/01k_5m"</span><span class="p">,</span>
<span class="s2">"description"</span> <span class="o">=></span> <span class="s2">"Lake Tahoe"</span><span class="p">,</span>
<span class="s2">"score"</span> <span class="o">=></span> <span class="mf">0.46974313</span><span class="p">,</span>
<span class="s2">"boundingPoly"</span> <span class="o">=></span> <span class="p">{</span>
<span class="s2">"vertices"</span> <span class="o">=></span> <span class="p">[</span>
<span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="p">{</span>
<span class="s2">"x"</span> <span class="o">=></span> <span class="mi">1092</span><span class="p">,</span>
<span class="s2">"y"</span> <span class="o">=></span> <span class="mi">555</span>
<span class="p">},</span>
<span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="p">{</span>
<span class="s2">"x"</span> <span class="o">=></span> <span class="mi">1273</span><span class="p">,</span>
<span class="s2">"y"</span> <span class="o">=></span> <span class="mi">555</span>
<span class="p">},</span>
<span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="p">{</span>
<span class="s2">"x"</span> <span class="o">=></span> <span class="mi">1273</span><span class="p">,</span>
<span class="s2">"y"</span> <span class="o">=></span> <span class="mi">944</span>
<span class="p">},</span>
<span class="p">[</span><span class="mi">3</span><span class="p">]</span> <span class="p">{</span>
<span class="s2">"x"</span> <span class="o">=></span> <span class="mi">1092</span><span class="p">,</span>
<span class="s2">"y"</span> <span class="o">=></span> <span class="mi">944</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="p">},</span>
<span class="s2">"locations"</span> <span class="o">=></span> <span class="p">[</span>
<span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="p">{</span>
<span class="s2">"latLng"</span> <span class="o">=></span> <span class="p">{</span>
<span class="s2">"latitude"</span> <span class="o">=></span> <span class="mf">38.940395</span><span class="p">,</span>
<span class="s2">"longitude"</span> <span class="o">=></span> <span class="o">-</span><span class="mf">119.91884</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="p">},</span>
<span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="p">{</span>
<span class="s2">"score"</span> <span class="o">=></span> <span class="mf">0.29812887</span><span class="p">,</span>
<span class="s2">"boundingPoly"</span> <span class="o">=></span> <span class="p">{</span>
<span class="s2">"vertices"</span> <span class="o">=></span> <span class="p">[</span>
<span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="p">{</span>
<span class="s2">"x"</span> <span class="o">=></span> <span class="mi">960</span><span class="p">,</span>
<span class="s2">"y"</span> <span class="o">=></span> <span class="mi">902</span>
<span class="p">},</span>
<span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="p">{</span>
<span class="s2">"x"</span> <span class="o">=></span> <span class="mi">1558</span><span class="p">,</span>
<span class="s2">"y"</span> <span class="o">=></span> <span class="mi">902</span>
<span class="p">},</span>
<span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="p">{</span>
<span class="s2">"x"</span> <span class="o">=></span> <span class="mi">1558</span><span class="p">,</span>
<span class="s2">"y"</span> <span class="o">=></span> <span class="mi">1073</span>
<span class="p">},</span>
<span class="p">[</span><span class="mi">3</span><span class="p">]</span> <span class="p">{</span>
<span class="s2">"x"</span> <span class="o">=></span> <span class="mi">960</span><span class="p">,</span>
<span class="s2">"y"</span> <span class="o">=></span> <span class="mi">1073</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="p">},</span>
<span class="s2">"locations"</span> <span class="o">=></span> <span class="p">[</span>
<span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="p">{</span>
<span class="s2">"latLng"</span> <span class="o">=></span> <span class="p">{</span>
<span class="s2">"latitude"</span> <span class="o">=></span> <span class="mf">38.943923</span><span class="p">,</span>
<span class="s2">"longitude"</span> <span class="o">=></span> <span class="o">-</span><span class="mf">119.928989</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="p">}</span>
<span class="p">]</span>
<span class="p">}</span>
<span class="p">]</span>
</code></pre></div></div>Simon Rentzkesimon@rentzke.comThe age of intelligent api’s is here.My new Hobby2015-10-26T17:20:54+10:002015-10-26T17:20:54+10:00http://simon.rentzke.com/my-new-hobby<p>I used to think I had a few hobbies, mountain biking, rugby, vege gardening, coding, etc, etc, but then my wife and I bought a <a href="https://en.wikipedia.org/wiki/Queenslander_(architecture)">Queenslander workers cottage</a> that needed a <em>bit</em> of attention.</p>
<p><img src="http://s3.amazonaws.com/simonrentzke/stream_images/images/17/original.jpg?1445843829" alt="queenslander renovation" title="Queenslander" /></p>
<p>I had zero interest in renovation. But this beast has taken over, it consumes every minute of my spare time, and has for the past 2 years.</p>
<p>We’re nearing the end of the whole experience, and although feeling exhausted, we’re really stoked that a lot of things have come together.</p>
<p>If anyone is interested here is our progress in pictures (apologies for the blogger blog :)
<a href="http://one-egbert.rentzke.com/">Egbert Blog</a></p>Simon Rentzkesimon@rentzke.comI used to think I had a few hobbies, mountain biking, rugby, vege gardening, coding, etc, etc, but then my wife and I bought a Queenslander workers cottage that needed a bit of attention.Internet Speed in Australia2015-09-24T14:01:05+10:002015-09-24T14:01:05+10:00http://simon.rentzke.com/internet-speed-in-australia<p><img src="http://24.media.tumblr.com/tumblr_m4dyxavPbo1qbkauho1_r1_500.gif" alt="Internet in Australia" /></p>Simon Rentzkesimon@rentzke.com