• Hong Kong after 7 months
    Going back to Hong Kong after 7 months for a work trip felt different. A lot had changed – I could sense how the city was trying to get back on its feet after the tumultuous events of last year. The national security law had all but stopped the protests. Spirits felt a bit dampened, with the double whammy of Covid-19 hurting the economy and the daily lives of people. The government, however, had done an amazing job at containing the virus and things were just starting to open up after the third wave. My stay in Hong Kong started […]
  • Stock Price Data Analysis – 2
    Let’s start by loading the data into memory first. The data is per minute stock prices for the SNP500 bucket for the 10 days prior to 17 Dec. This means we have 390 x 10 so about 3900 rows per stock. This, however, is not absolutely true since some data has holes in it – i.e. some stocks are missing some minute values. This needs to be taken care of after loading in the data. Data sanitisation can sometimes take a large part of the time one spends on data-analysis. What I do here is to first only select the […]
  • Interpretation of PCA Results
    An intuitive interpretation for PCA results
  • Stock Price Data Analysis – 1
    Introduction I recently was able to get my hands on raw stock prices data (open, high, low, close and volume) for the Standard And Poor’s 500 index (SNP500). The data was for each stock in the SNP500 bucket (actually 504 stocks since some of the stocks are listed twice for different types of shares issued) at two types of intervals: Daily (from each market open date since Jan 1 to Dec 15 of 2016) Per minute for each of the 390 minutes the market is open for the past 10 days since 17 Dec ’16. This gave me an opportunity […]
  • A look into Pakistan’s Education Situation using Machine Learning
    I was looking to experiment with some of the machine learning techniques I had been learning the past few months. I was particularly interested if I could glean any insights about the education system in Pakistan. Despite the efforts of the government, a large proportion of the nation is illiterate and the state of education delivery remains poor. I am of the opinion that a very focused data-driven approach would be a better method of resolving the crisis that the country faces than spending tons of money on the distribution of laptops. There are many problems which plague the schools […]
  • Social Injustice and the Individual’s Moral Obligations
    Individuals and their moral obligations to defeat social injustices in society or immediate vicinity.
  • On the Integration of Immigrants in Society
    Are secular societies better at assimiliating immigrants compared to not so secular ones?
  • Victor Frankl: Suffering and Meaning
    Viktor Frankl was an Austrian neurologist and psychiatrist as well as a Holocaust survivor. In his book – Man’s Search For Meaning, he writes about the psychology of the prisoners in the Holocaust concentration camps. I very recently read a passage on suffering (and what meaning people can derive from it) in Frankl’s amazing book. Perseverance in the face of suffering is a subject I have given significant thought to, and Frankl eloquently summarises some of the most important aspects of suffering and its meaning in life. This was by far the most profound piece of literature I have read […]
  • Violence and its Effects on Intergenerational Mobility
    Up until a few days ago I was not fully aware of the implications of systematic discrimination and bias against a particular race or ethnicity on its future generations. A lot has been said about how assimiliation of people from races or ethnicities discriminated against in the past into the mainstream economy is an arduous task. I used to think of all of this as pointless. To me the idea that people would blame the current living conditions of their people on their history was absolutely ridiculous, given the time period between them and the historic discriminations which had occurred. […]
  • Child Watches, Child Learns
    I was sitting on the upper deck of the tram on my way to a friend’s place earlier today. It was raining heavily and it was a pleasure to hear the pattering outside, but my attention was drawn towards the very front. A very agitated kid of around 18 was standing in front of his seat and seemed to be in a trance – he would glance at his watch, then peer outside the window and then look back at his watch over and over again. In my curiosity, I looked fixedly to what would have been very rude staring. […]
  • Using OpenCV to use great circle plots for horizon detection
    Now that we have a way to plot great circles on to equirectangular planes, how do we use that to detect the pitch and roll of the camera and consequently stabilise the video? Draw Curves First, we need to draw the curves as opencv images: We calculate the curves according to the roll and pitch provided. Then we normalise the curves such that when they are drawn, they appear from 0-2pi and 0-pi on the x and y axis respectively. Drawing these points as polylines using opencv generates an image like this. 10° roll, 15° pitch Using a script, we […]
  • Python for Great Circle Projections – Part 2
    Python to plot projections of great circles on to equirectangular planes
  • Python for Great Circle Projections – Part 1
    Python to plot projections of great circles on to equirectangular planes
  • Password Recovery Best Practices
    Some insights on implementing a password recovery system securely.
  • Event-Driven Design for DOM Manipulation
    After one of the projects in my company had reached a point where refactor was not only recomended, but necessary to clean up the code and make it maintainable, I decided to use a different approach for DOM manipulation in the client facing web app. Since we were doing a major refactor (yes, it does exist!), I was given considerable leeway to choose a different way of dealing with the big problem of manipulating DOM elements. After considerable discussion, we decided to go for an event-driven design. I decided to use Node’s event class as our event handler, and re-designed […]
  • Javascript Web Application Development Flow
    Recently, I was asked a question about Javascript development: how are modules used in ES6 and then how do they magically become one .js file when deployed. Since I have been using a workflow which answers both of these, I am going to write it up here for anyone who wants to know. Big projects can be written in modular form in one of the EcmaScript standards (Actionscript, Javascript, Typescript etc) and then compiled into one file for web delivery. The current project I am working on is written in ES6, the 2015 Javascript standard which natively supports modules. This […]
  • Terrains using Three.js and SRTM Data
    Let’s jump right in to three.js plane buffer geometry. Three.js offers a buffer geometry object which is a simple array of height maps, much like our binary file we used from the SRTM data. Compared to the normally used plane geometry, buffer geometry uses much less overhead and is significantly faster if the object does not need to be modified once initialised. To make things simple, I will assume that the terrain we need to generate is contained only within one binary hgt file, i.e., the terrain is within one latitude and longitude cell (this means that the bounding box […]
  • Using SRTM Elevation Data in Javascript
    1 arc second SRTM Data was made [publicly available] (http://www2.jpl.nasa.gov/srtm/) by NASA a few months ago. SRTM Data is collected by the Shuttle Radar Topography Mission. It sweeps the surface of the earth to find irregularities which form the height map. According to the mission website: To acquire topographic (elevation) data, the SRTM payload was outfitted with two radar antennas. One antenna was located in the shuttle’s payload bay, the other on the end of a 60-meter (200-foot) mast that extended from the payload pay once the Shuttle was in space. For our application, we were using SRTM Data for […]
  • WGS84 Projections
    A short tutorial to do localised projections for WGS84 (lat,lng) data
  • Perl File Management
    Simple Perl File Management App