🌍; view from the web

by Ricardo Tavares

The mixed bags of online events

I imagine that a lot of people have experienced much more online events in the past year. Now, alongside the wide variety that events offer by their very diverse nature, there are also a lot of different tech solutions that people reach for when putting their event together. I've seen online events that are very e-mail driven while others rely on live chat. Some are pretty much just a video playlist published at a specific time while others focus on interacting with the audience at the lowest stream latency they can broadcast. Some events leverage all they can from the open web while others close themselves off inside this or that social network.

There's also an important balance between offering a cohesive experience that can be seen as a true meeting point in people's minds and the need to have all the features expected of your online event. This landscape stretches out according to your familiarity with different technologies and how easily your audience can move between them. However, even if everyone has no problem jumping between platforms, each jump can jeopardise that feeling of a common digital space. I've experienced events in which the only thing reminding me of "where" they took place was the graphics and the music between speaker presentations. Otherwise, it can be easy to feel lost between something like a wall of e-mails, a landing page with a schedule hammered between sections, a couple of streams on some free video platform, a chat server on yet another platform, plus questionnaires in some other site, video-on-demand in another... events like these are just a blur. If they don't strive for some unique unifying context, online events can become quite forgettable. The one thing that can always shine through is the quality of each speaker, but event organisers probably want to generate some synergy beyond that, whether they're selling tickets, exchanging knowledge and/or rallying a community under one vision.

Fortunately, I've also experienced online events that had some nice balance between providing context and providing features. You can check out my article about my first FOSDEM here, for example. I believe you can get away with stepping outside your branded platform once or twice when you've established at least one satisfying core loop, one simple story in which the attendee is the hero. For example, in FOSDEM, you could watch every session live in a chat room dedicated to that track and then the speaker would answer questions or comments from that chat. Another helpful sequence can be bookmarking a session on the schedule and receiving a notification when it is about to start which links you back to that session where you can start watching it. Or it can be just about a sponsored session that reminds you to check out that sponsor's page inside the platform while the stream keeps playing in the corner of the screen. Indeed, it's this feeling of movement across digital space or time while still inside the context of an event that provides meaningful engagement. I'm sure that, as we move back to physical venues, online events will also learn to do better.

If you liked this article, you might want to subscribe to the RSS feed, maybe follow my Twitter or learn more about me.

Passwords cross all boundaries, how can we manage them?

Passwords are a great solution and a huge problem. People use them every day, not only as a way to claim ownership of services and products, but also to share that access with other people they trust. Passwords are great because they stand outside everything. You're not forced to have service A in order to access service B. Passwords are portable, platform agnostic and not tied to a particular identity. Allow me to stress this aspect because you'll not hear it mentioned by corporations that are interested in locking down account ownership completely: people share passwords with their close relations. One person pays for something and then they can easily share access to it by simply giving out that password. Account details are often sent in plain text e-mails, but also in loosely encrypted chat messages. The security risks are considerable, but there are also hidden benefits to using passwords as opposed to biometrics or device keys. People who push for alternative solutions like to pretend that this is not a welcome feature that everyone sooner or later depends on.

However, passwords are bad because they're powered by cognitive load. You need to generate, remember and keep secret some strange and unique series of characters. And some other person needs to technically secure a matching hash of your password. And each time that we fail at this task, a permanent record of leaked passwords is eventually made available worldwide. Finally, each of these failures can be relatively inconspicuous, it can be possible that right now someone is taking some advantage of your leaked password and no one will ever know. Indeed, handling passwords is not a fair task. You're supposed to do it by yourself and, if there's a problem, it's possible you'll never know until maybe some money has gone missing or some unsuspecting account comes up associated with criminal activity. No pressure.

Therefore passwords are a good example of how real-world security is a balance between what features you value and the risk associated with each of those, it's always a mixed bag. If we recognise that there are both benefits and costs to using passwords, we can consider accepting and mitigating the risks of having our accounts compromised. Password managers are not a perfect solution, but they are currently the only way to keep passwords around with the degree of freedom they offer. Like other flawed solutions, password managers don't solve the problem but rather kick the can down the road. But, for those unaware of this solution, I'll describe a complete password management service that you might even want to pay for.

The service still depends on a single master password that you create following all the usual best practices. This one good password is how you access all your other passwords that the service can then generate, store and type for you wherever you need them. That's why this has to be seen as a complete solution for all your accounts across all your devices. It involves a website, iOS/Android apps and extensions for most browsers, all so you can create and use secure passwords everywhere. I do recommend not going for half-measures. If you're going through the work of having a password manager, you want it to remove 99% of the burden of juggling passwords in your head. And the reason why the best service is probably a paid one is because having just one user interface to manage your passwords only gets you halfway there. Now you want it to automatically type into login screens inside apps on your phone. You want it to save a new account you've just created in a new browser you've decided to try out. These cross-platform features unfortunately cannot happen for free. Regular development time has to be spent chasing down the inevitable changes in browsers and ecosystems. The kind of drudge work that free and/or open source software rarely finds people to do. But it all adds up to providing a user experience that can greatly improve your security. It's often forgotten how important an easy UI and a good UX are to solid security practices.

And here are some not-so-obvious benefits to using a complete password manager:

Some platform you signed-up for got hacked and now you're forced to change your password. That's easy.
This one torpedoes the usual method that people have of using some personal cypher to generate passwords from the name of each platform. For example, your Google password would be elgoog.2001 and your Yahoo one oohay.2001, for example. Now, Yahoo gets hacked and what do you do? This specific account now deviates to oohay.2002, right? Not very secure or easy to remember. With a password manager, you don't care what the password was or what the next one will be. You're not particularly bothered even if some company forces password rotations every 3 months. Just generate a new one and get on with your life.

This specific website has some strange password requirements. Not an issue for you.
Another cannonball shot at the starboard of personal cyphers. Now, let's say your bank requires two capitalized letters. Following on the previous example, you have to do something like KNab.2001,right? Again, not something we should be wasting brain power on. Password managers allow you to generate randomized text that can match any of the usual or most annoying requirements.

You can turn the magic dial up or down at your convenience.
When you switch to a password manager, you're not forced to change all your old passwords. It's an incremental solution. You can start by keeping all your not-so-secure passwords and then gradually replace them over time as you get the chance. You can also account for situations where you don't expect to have your password manager installed by generating a long pass phrase that's easy to copy by just looking at it. For example, you can check it on your phone to login in a new computer.

Sites you want to use keep asking you to create an account. An easy process.
When you don't use a password manager, you tend to shy away from creating accounts because it's another password you have to create and remember. In a way, this service plays the same role as having a virtual credit card. You're more comfortable with getting things done online because there's always some protective layer. If some website shows itself to be insecure (like by showing you they keep your password in plain text), you can cancel your account without having exposed some personal cypher you use or a password you use somewhere else. Sometimes you can even take an extra step to anonymise your account: you can generate both a random username and a password that have zero relation to any of your online identities.

I don't want to give any specific recommendation for a password manager, but I personally use 1Password and at work we have LastPass. I've also heard good things about Bitwarden which you can self-host. Like in anything involving your own security, it's best to do your own research up to a point where you make an informed decision. So thank you for reading!

If you liked this article, you might want to subscribe to the RSS feed, maybe follow my Twitter or learn more about me.

Pseudonymity: are we to walk down digital streets with our names tattooed on our foreheads?

When we arrive late to a meeting where people expect us to say something, most of us listen for a while before trying to contribute. Eventually, we may realize that the meeting has progressed far beyond what we are able or willing to understand fully. And yet there's a need to come together, so what do we say? Some people can be the first to speak out by disregarding what they don't know and focusing on the obvious potential solutions. By doing so, they can derail the rest of the meeting, specially if more people keep arriving late. This is an allegory for the state of the internet right now as affected by politicians and corporate media. They've arrived late to the party and never cared to understand what it was about in the first place. In a way, that's fine because they were dragged into this by a massive smartphone adoption, also filled with people who would very much prefer to ignore the challenges presented by this new technology.

One of the main challenges is nothing new, it has only been accelerated by tech that keeps advancing faster than ever in human History. It's the challenge of losing context, having no common reference of space or time. You can pick up a book and read it anytime and anywhere, you can delve into the text within a context that can never be predicted by its author. You can exchange letters with someone and you never know for sure when they will arrive or how many times they will be read or in what circumstances. You also can't be 100% certain who wrote that text or who is actually going to read it. The internet offers very much the same issue but literally without giving us the time to think about it. And the speed and frequency of our communication also affects the quality of the content itself, which means that the message becomes more dependent of a context that lacks our usual frames of reference. We don't know who is really on the other side. We're not there when the content came about. The written word, illustrations, photographs, audio recordings, video, network computing, live streams, it's always the same challenge coming at us faster and faster.

Mafalda by Quino

Up to a certain scale of interaction, the content can stand on its own without much context. But many people would not go to that internet, the one where only ideas matter and you can be anyone or anything all at once. That's too dorky, that can't be real, people seem to instinctively stay away from these abstract domains. So the obvious solution, the one that people arriving late to the internet keep reaching for, is to get as much personal data as possible associated with your internet presence. And that is a possible answer to a lack of context in our digital content. We have no idea where or when a piece of content came about but we can try to label it with something close to a real name, a real photo, etc. Again, this is not a new strategy, books have been plastered with photos of their authors for decades, but the tactics have now evolved to leverage the reach and immediacy of these new networks. You may not want to have anything to do with computers, but photos of your newborn grandson are being posted there right now, so of course you're going to open that black box and take a good look inside.

Other approaches to providing additional context don't scale just as well as putting people in well-marked boxes with their names on them. For example, some digital platforms try to grow their user base out of some local context, like university campuses, busy street corners or meet-ups for all kinds of hobbies. In those cases, terms of service don't usually require so much personal data to identify users. People already choose to share just enough information about themselves so they can be identified in local meet-ups if they want to. Or it may be that the purpose of those platforms is to indeed provide an anonymous venue for an already well-defined community. It can make sense so that people feel more comfortable pointing out things that need to be fixed in their city or campus. But this usually requires considerable work moderating content and it only has a kind of supplemental value, it works in parallel with your analogue life. This does not generate the kind of scale where the app already comes pre-installed on your phone. Bigger networks tend to get bigger while smaller networks eventually get smaller.

The context provided by having everyone pinned down to one identity is self-evident in its value. You are there because everyone in your life is also there. Your boss, your family, your childhood friends, all the contacts on your phone. All the boundaries in your life can become blurry in a context with no space or time, only people as they perceive you across every facet of your life. Of course, this happens in gradients across the world and depends on how people exercise their individual freedoms. In some countries, indeed you are forced to use the internet as an identified citizen subject to social monitoring. In others, you may have some freedom but perhaps you can only access the internet through some corporate gateway where you're a clearly identified consumer. But in many countries, you have a choice of different ways to get connected and you can use the internet as a free citizen. You don't have to limit yourself to a single identity that matches what's written on your passport. So there can be a lot of nuance in how many online personas you build and how they are attached to unique identifiers like your phone number, a photo of your face, your age, name or place of work. You can enjoy going surfing or spending time with your kids without being fitted into just being "surfer girl" or "stay-at-home dad". As Erving Goffman has explained in his book "The Presentation of Self in Everyday Life", "identity is not a singular thing; identity is a role people play that shifts as audience and other contextual factors shift. The 'self' people present is never a full representation of who someone is, nor is it a fixed identity that cannot shift as other factors shift. After all, most people would act one way on a Friday night out with friends and another way on a Sunday dinner with family."

But even when we can have all the freedom and privacy in the world, if we leave those advantages on the table, governments and cooperations like to push those away from us, since propaganda and advertising don't work so well on moving targets. And besides, the opportunity to turn every screen into a public diary or a shop window is too attractive for them not to explore whether it's possible. It's up to us citizens to realise the power of the internet as a tool that we can also forge for ourselves and not just within the parameters set by those who are in the best position to manipulate everyone else. It's like when companies tell us we are saving money when we are spending money on their products. Thinking for ourselves is irreplaceable. If we don't, somebody else will try to do the thinking for us.

But let's instead put convenience above all else and see what the price tag is. If we completely sacrifice anonymity for the sake of providing some familiar context to our digital networks, what do we have to lose? The thing is, once we establish this premise, there are problems that materialise down the line and we find ourselves constrained in our ability to solve them. For example, once we establish that your face is part of permanent records in social networks the minute that you're born, once that anchor becomes so heavy, problems like cyber-bullying and general mental health become much more serious than they should be. Once you're locked in to being always online and always with the same identity that's so tied up to how you see yourself, you are much more vulnerable to attacks that can ambush that persona. Once your online presence determines your ability to get a job and that identity is all you have, you may think twice before voicing political opinions. Once we allow platforms to prioritise daily content that has as much personal data as possible (vlogs, stories, IRL drama, etc.), all content creators are pressured to expose their intimate lives to please ad-driven algorithms. All these cases of context collapse, a term proposed by Alice Marwick and Danah Boyd (2011), should not be mistaken for an increase in authenticity. It's simply a tendency to smash any possible context that you have together in an attempt to make it feel more real.

Indeed, if you just let go, it all clicks together as you slide into your pre-designated and clearly labeled consumer role. And although the rule of convenience can fold very well into growing addiction, going down what seems to be the easy road can feel much more comfortable. In an article by Emily van der Nagel and Jordan Frith entitled "Anonymity, pseudonymity, and the agency of online identity" (2015), they agree that "certainly, there is room for negativity and antisocial behaviour in spaces that allow people to interact without showing their faces or 'real' names." Furthermore, "understanding anonymity as the cause of such deviant behaviour is an attractive prospect, as this also identifies a simple solution to combatting incivility online: get rid of anonymity." However, "to take away, or even stigmatise, anonymous communication by moving towards a 'real name' Internet is to shut off important avenues for productive identity play, self-exploration, and behaviour contextualisation online." Therefore, "while safety concerns about anonymity are real, it is also true that real names can make people feel less safe and can inhibit behaviours they engage in online." We should keep arguing that "practices of anonymity and pseudonymity may be complex, but they add texture to being social on the Internet. The option of not using real names online allows people to control what they reveal about themselves and who they reveal it to."

The value of pseudonymity in the internet is part of how we lay claim to it as a public space. It matches how we see other people walking down the street and they see us. It's part of a continuum of anonymity in which we can have all kinds of satisfying interactions without exposing ourselves more than what we feel better serves that particular context. It's more real than 'real' names and it can enable true authenticity. So we should not let these hasty solutions monopolize our meetings and stop the conversation for the future of the internet, one in which every identity needs to have a voice.

If you liked this article, you might want to subscribe to the RSS feed, maybe follow my Twitter or learn more about me.

I tried very hard not to like my first Macbook

Due to personal circumstances that forced me to pull the trigger on my decision to eventually get a good ultrabook, I recently got the M1 Macbook Air. I'm quite happy with it, but not so happy with having purchased it. It was a combination of being pressed for time and having few viable options for my country of Portugal. Readers of this blog already know a few of the problems I have with Apple and I usually do like to vote with my wallet. Nevertheless, this laptop matches very well with what I consider to be a good ultrabook. It's a compromise.

M1 Macbook Air

My experience with Apple products has never been pleasant. I remember receiving an iPod Nano from a close relative and weeks later letting them use it instead. The absence of any customisation has always confused me as a user, makes it harder for me to do anything, not easier. I later had to interact with the Apple ecosystem as part of publishing apps to iOS and the experience hasn't been any better. Again, Apple fights you all the way if you're not using the thing to do the thing as they expect you to. If you only have a Mac Mini to compile apps, it's gonna need an update when you need it to work. If you need to test different apps in different devices, installing all of them is a pain.

Meanwhile, I had been looking for what I like to call "a real laptop", one that can easily be useful as an everyday carry. In my case, useful for development, design, working on the web and not so much gaming (although I love playing games). Theoretically, my options could range not only between the Thinkpads and the Dells, but also include the latest models from Microsoft, Asus, LG or HP. And let's not forget Linux laptops like the ones from Slimbook or Tuxedo. But in practice, a couple of important points start shooting down a lot of these options. For work, I really need a screen that's at least 16:10 or 3:2 and a keyboard layout in my native language. Also, in Portugal it seems that the only companies that actually sell these premium laptops themselves are Apple or Microsoft. And going through resellers in the middle of a spike in laptop demand means you either get price gouged or the model you're looking for isn't available anytime soon. While on the other hand, distribution wise, clearly the one company that really wants to sell you a laptop in Portugal is Apple. Their site has the thing, you buy it and they'll deliver it. This is similar to Microsoft but on a much more commited level.

Still this is not about making the choice that is possibly less bad. Until recently, I would simply never buy a macbook given its uncomfortable keyboards and questionable value for money in terms of performance per wattage. And the lack of repairability is still very much an issue. But the M1 Macbook Air changed that by leaning into what Apple does best. Yes, it's still a Mac but it's also kind of an iPad with a keyboard. Meaning, huge battery life, zero fan noise and the best performance you can get under those restraints. A lot of laptop manufacturers and reviewers are still playing in a different field, they are selling gaming rigs disguised as student laptops so parents will buy them for their kids. Or they are B2B solutions that I can't hardly access as a consumer in my country. Or they're cool independent shops but with chassis that seem to lag at least a year behind major brands. I've personally used several laptops in my life but, up to now, they were never truly portable. Even my Microsoft Surface Pro required charging it throughout the day and had issues with sleep mode. This Macbook Air weights almost just as much, the charger stays home and I almost never shut it down.

The final hurdle that I couldn't be bothered to jump over was MacOS. Only used it at work and I still think that it has a very weird learning curve. You either do exactly as Apple wants you to or you're suddenly confronted with hidden hotkeys and terminal commands. Still, learning a new OS is not some gigantic task, I just had to be willing to invest the time (I've used every version of Windows and a few Linux distros). Unlike iOS, you can still change a lot about your MacOS setup. And one thing that makes it usable that also matches my long-time workflow, is using it with the trackpad and putting everything in full screen. I personally don't like splitting the screen too much and prefer switching between virtual desktops. The other thing that MacOS kind of requires if you want to make it work for you is deliberately paying for software. Again, I don't have an issue with that, specially when you can still make a one-time purchase and get the full application.

These are still my first impressions, so I'll just end this very subjective purchase review with a few things that surprised me with my current setup and my first Macbook in general:

  • The zsh command line is always just a key press away (iTerm2).

  • It's easy to monitor temperature, network usage and other system stats from the menu bar (iStatsMenu).

  • Passwords can be very accessible with the fingerprint reader (1Password).

  • Non-native electron apps like Discord have the worst performance... and yet I've been in a group video-chat for a whole afternoon and the battery was still above 10%.

  • Vivaldi is actually a good option on MacOS since you can move your tabs to the bottom of the window which, in full screen, avoids having the menu bar pop up every time you reach for them. On the other hand, Firefox is noticeably slower in this OS.

  • The screenshot tool that can also record your screen is actually very configurable and definitely better than the one on Windows.

If you liked this article, you might want to subscribe to the RSS feed, maybe follow my Twitter or learn more about me.

Hybrid events and my first FOSDEM

Events moving towards online has become one of the definitive trends of the decade. Not that people want to stay home, but once we've been forced to, the advantages become evident. Any event can reach out beyond its usual physical time and space to bring more people together. Even if nothing can beat the hallway track of a fully-present context, I don't believe the online facet of large events can just go away now, specially when video-on-demand allows us to go back to any content we'd like to watch. The technology was always there, it just didn't seem so relevant as it has become now.

In a lot of ways, FOSDEM is an unique case among big annual events. As it says on this year's website "every year, thousands of developers of free and open source software from all over the world gather at the event in Brussels. In 2021, they will gather online." And they did, maintaining a long tradition of filling countless tracks with an overwhelming variety of talks. Usually, FOSDEM fills rooms to full capacity as anyone can drop by to check out the event, it is both a free experience and an exercise in freedom. This year, instead of testing the capacity of its venue, FOSDEM tested the limits of its online infrastructure, specially during the first hour of the first day. It was a rocky start but one to be expected: an undetermined large amount of people arrives all at the same time to set up a profile, browse tracks, open streams, live chat... I'm not sure if people realize the unique value of having a huge event you can attend without giving out any personal details before hand.

I've always been curious about the event, so this online edition was a no-brainer. By creating an account a few days before the event, I avoided that bottleneck and only had trouble playing live streams in the Element Matrix client for the first couple of hours. The tracks I chose to keep tabs open on were the main ones plus Mozilla, Javascript, Open Source Design and Real Time Communications.

Screenshot of my online FOSDEM experience

Here are a few talks that I can recommend from watching them live:

And here are a few that I've watched through the VODs (which are an essential feature in the case of FOSDEM):

Looking towards the future, I am definitely more interested now in attending FOSDEM in person. Watching the event online does allow you to keep an eye on several tracks all at the same time and to jump between them depending on which session is more engaging. But I'm sure that the in-person experience can have an unique interpersonal value that is very much a part of free and open source software. This is a kind of software development that is all about people coming together, with or without a physical context. By always thinking about their audience on a larger scope, I hope that indeed many events tap into the potential of having both a physical and a digital venue.

If you liked this article, you might want to subscribe to the RSS feed, maybe follow my Twitter or learn more about me.

iOS devs working as unpaid salesmen for Apple

One of the ways through which Apple has posted new records in their services revenue is by having you buy a developer account and then forcing you to make other people buy developer accounts. How does that work? Well, let's turn this around and say that you're not a developer in any shape or form. Your job is, I don't know, geologist or something.

You're the president of the geology society in your country. And every year you have a couple of meet-ups that involve a few thousands associates, students and maybe some sponsors when you're lucky. Like in every other similar institution, you organise all of these as a side-project, a lot of it is done up to the last minute, but as long as you keep everyone posted on what's happening, all tends to go well. You already have a website and a newsletter, but your members would also appreciate having real-time information of all those last minute changes that happen in your events. Plus, your website kind of sucks on phones, so you're thinking of getting a mobile app for your next event. Let's say you can do that with the help of some sponsor. You contact an app developer and ask them what do you need to get some push notifications going out to your associates. They need to ask you a few questions before answering that.

You answer "of course" when they ask you if you need both an Android and an iOS app (most of your associates wouldn't even understand the question). And you answer "that's not an option" when they ask you if you'd like to feature your meet-up inside their own multi-event app. Your sponsor wouldn't accept not having an app with the brand of your geology society front and center. Plus, installing some random event app would make the whole process too confusing for your associates. It seems the app developer has had this conversation many times already with clients in the same situation. They explain that you need to buy a developer account from Apple. What? Yes, it all sounds much more complicated than what you expected. Apparently, your iOS app has to be published by the content owner, meaning the geology society itself. You tell the app developer you know nothing about having anything on the app store. They say it's OK, they were forced to go through this process many times already with other clients and they can take care of everything if you follow a few critical steps with Apple directly. They're not making any money with this process, it's just the only way that Apple eventually allows you to have a push-notification with your brand on it. Well then, guess you're a developer now...

People who have ever used App Store Connect probably remember there's a little drop-down menu in the corner for any possible accounts associated with your profile. Years ago, there was not much of a reason to click that, you had your one account and the big My Apps icon for all your stuff. Now, our friend Tim Cook has found a way to generate revenue by populating that drop-down menu for many iOS devs that publish apps for clients. The same apps are getting published, but now the devs have to do that little song and dance explained above to essentially move them from the big My Apps icon into dozens of new developer accounts they need to control now. Never underestimate the power of capitalism to make more money for shareholders in exchange for keeping things pretty much the same while generating useless busy work.

Still, this is not even last year's news. As always, any corporation will keep pushing for as long as they're allowed to and, since the drive towards alternative revenue sources is working, they won't stop going. Apple is not only continuing to require developer accounts from non-developers, but also taking strides to "uberize" iOS devs by finding every possible excuse for in-app purchases to be used or else your app gets rejected. Apple also lags behind with supporting web apps as an alternative to the App Store when a lot of what you find in the iOS store could work entirely from the web. And Apple has this dream that your geologist society is going to buy an enterprise account to be able to deploy private apps only for their members, that's yet another potential cause for rejection.

Overall, there's a lot of risk associated with publishing in the App Store that comes from Apple themselves. Unless you aim to build exactly the kind of app that they can easily categorize and monetize, you have to think carefully about investing in the platform. And, with the drive towards everything not getting payed for with money, that problem is exacerbated by the lack of viable competition from other ecosystems. It's like a choice between the gig economy or unemployment. We have to break outside this box.

If you liked this article, you might want to subscribe to the RSS feed, maybe follow my Twitter or learn more about me.

PHP Upgrade Story: Four Lessons Learned

Back at the day job, we have your usual PHP code base that runs a lot of the business and we needed an upgrade from 5 to 7. Business involves a variety of small to large services plus different one-time projects that may require maintenance. So, upgrading had to be a gradual process of isolating a domain that could be upgraded, making/testing changes and benchmarking PHP performance. I've already written about a specific challenge with database encoding that was probably the most difficult and time-consuming to overcome. You can read all about it here. This time, I'm looking back with a more general look on managing different PHP versions with cPanel's MultiPHP approach. MultiPHP edits .htaccess files and maintains different php.ini configurations, making it seem easy to jump between versions with just a few clicks. As one might expect, things are never so simple. Here's what I learned:

1) Be Warned

Starting with the more obvious one, the history of PHP moving forward is one that goes from seeing you shoot yourself in the foot and showering you with guns to tapping you on the shoulder and letting you know what you might be aiming at. For years, PHP has let you push code with potential problems that later on may explode in ways that are hard to pin down. With newer versions, PHP becomes more strict, so the easiest first step when considering an upgrade is to turn up your errors. Let those warnings all come out so you can investigate any type mismatches or assumptions that are unsafe to rely on. This already makes your code more resilient even before upgrading.

2) I Am Root

Onto a more practical matter, MultiPHP does nothing about your cron jobs, so you need to manually configure them to call on a specific php bin like /opt/cpanel/ea-php73/root/usr/bin/php instead of just php. However, as you do so, you will find out that the default working directory will be the root of your home and not the folder where your .php file resides. This may have lots of unexpected consequences if you're relying on relative paths. You should probably change those to absolute, but for simple scripts you can get away with simply adding something like chdir(dirname(__FILE__)) to maintain the same behavior.

3) Runs In the Family

If cPanel is just another tool that you use while still setting up your own folders in your server, eventually MultiPHP is lying to you when it says you are assigning some version to a specific domain. What it does do is write in an .htaccess file at the root folder of that domain, which means that anything you have under it will be affected by that configuration, even if being called from a different domain. This normally shouldn't be a problem as some projects you might want to isolate probably have nothing to do inside the directory structure of another domain.

4) Apples to Apples

I've been using artillery.io to benchmark web applications, so I also wrote a few tests to monitor possible differences in performance with the upgrade from 5 to 7. If you try this after applying a default upgrade to some domain, you might be surprised to see much worse performance. What's happening really is that the new php.ini is probably not allowing each process to have as much memory. You need to check whatever variables are relevant (like memory_limit) and manually match each one so that both versions are indeed running under the same conditions. Depending on your workloads, version 7 should indeed perform better than 5. In my case, the tests that matched our use cases were very much dependent on database performance, so the difference in speed was not considerable.

With PHP being used by about 80% of all websites with a known server-side programming language, I hope these few tips may still be of use, specially for people upgrading out of 5, which is still a considerable percentage of installations. Thank you for reading and I wish you all the best if you are working on one of these legacy code bases.

If you liked this article, you might want to subscribe to the RSS feed, maybe follow my Twitter or learn more about me.

My Love Story with Podcasts

Playable on-demand broadcasts (podcasts) are a simple case of people using open technologies to come together over a new medium that should be accessible to everyone. They are my favorite example not only of how useful RSS feeds can be, but also of the evolution towards audio files that are small, sound good and can be easy to catalog. Pretty much anyone with a laptop can record a podcast, the challenge as always is distribution. But the thing about just having a file that people want to listen to is how flexible it can be. You can put it on a website, send a link through a newsletter, have a full archive inside a torrent and no listener needs to have an account on any specific platform or even has to be online to listen.

My own history with the medium began about a decade ago when I was looking for independent content about games, namely tabletop RPGs. I became fascinated with the idea that you could kind of make your own personal radio to listen at your own pace anywhere. And also by the feeling of connection you get with people who are not very much different from you. They just sit if front of a microphone and talk (and then maybe edit the recording for hours, but let's not ruin the magic). And although some radios still have an experimental side to them, most of them have long become sequences of playlists and ads, while many podcasts still retain the ability to surprise you in every episode. More than that, some series can even be considered timeless, they are a contribution to the public library of an open internet.

After about a year of listening to podcasts, much like how people who read books eventually try to write since it's such a accessible craft, I created my own podcast and developed a taste for all the work that goes into it. For about four years, I committed to publishing something every two weeks and stopped after a long season of 101 episodes. Being dedicated to the niche of tabletop RPGs, I was the only podcast in my country on that topic, so I tried to cover just about every approach possible: reviews, interviews, essays, recording sessions, live panels, original music, round-table discussions, etc. Knowing zero about preparing a script, sound engineering or audio editing, I'm still just an amateur but I did learn a bit of how and why some podcasts are the way they are. Namely, the huge difference it can make having a team instead of going solo or having a good microphone in a decent room as opposed to trying to clean it up when editing. So maybe it's not that accessible, but the most basic setup sure can work for at least a few dozen episodes. And you still get to grow an audience, I made friends across the Atlantic Ocean, people that I would've never met if not for the podcast. This medium can truly embody the best of what an open internet has to offer us.

If you're looking for recommendations, the classic starting point that's mentioned when people want you to take podcasts seriously is Dan Carlin's Wrath of the Khans series from the Hardcore History podcast. You definitely can't go wrong with that (although I believe it's behind a paywall now), but I can also mention some of my personal favorites. Tech related, I recommend subscribing to Reply All (website, RSS feed). It stands out not only for the level of audio production, but mainly for putting humans at the center of how we deal with technology. For a completed series that may have timeless value, I can recommend More Perfect by Radio Labs (website, RSS feed). As a non-American who sees the value in understanding American history, learning about the evolution of the constitution and the supreme court is surprisingly useful. Finally, another easy recommendation if you ever thought about the importance of design in our lives, is 99 Percent Invisible by Roman Mars (website, RSS feed). Another great example of how to tell stories through radio that is playable on-demand. Trust me, this is a completely different experience from trying out audio-books.

However, like many other open platforms, podcasts also are what we make of them. And it's not like there's some clear corporate branding touting the advantages of playable on-demand broadcasts. Each podcaster really can do whatever and each audience member maybe finds out what a podcast can be and looks to see that promise fulfilled. Not surprisingly, what ends up happening are things like listeners who are content with going to a website, clicking play and keeping that page open to stream an audio file they could have downloaded and listen anytime anywhere. No wonder they have ended up limiting themselves to Spotify. An even more tragic misunderstanding is people who thought you needed an Apple device to listen to podcasts and in their minds have forever stuck an open platform into a closed ecosystem. Now, I'm not sure if I believe in love-at-first-sight, but convenience-at-first-sight sure is a popular all-or-nothing approach to technology. As if everything had to be right here right now for it to exist. Is convenience the only personal value that guides our actions? That would be a digital world without distance, one in which all things have to be targeted and pushed towards us since we won't walk towards them or even just pull them to use in our own time and with our own hands. So, it's only partly true, it's how things are for a lot of people who did not yet have the time to engage with the technology. We're still talking about podcasts, but it's also pretty much the history of the web, for example.

Anyway, if you're looking to get started, the easiest way to consume podcasts is to pick an app for your smartphone that does the work of managing your RSS feeds, namely automatically downloading the latest episodes so you can listen to them offline at your leisure. I have used BeyondPod and Pocket Casts, but now have switched to Podcast Addict. The best podcast clients inevitably become paid apps, ask for subscriptions and/or are assimilated by corporations that are interested in controlling the access to this open platform. Just remember that, at the end of the day, you are just going through a list of RSS entries to download an .mp3 file. That's what you want, not some closed source live stream that injects ads into your episodes. But, by the way, you should also realize that this remarkable ease-of-access also means that podcasters have very limited knowledge of the kind of engagement they are having. They basically know how many times an episode has been downloaded and that's it. So, if you enjoy a podcast, consider telling those people that you love their work and give them some feedback. It does make a huge difference.

If you liked this article, you might want to subscribe to the RSS feed, maybe follow my Twitter or learn more about me.

Our online lives lack context, they should also lack friction

It's an understatement to say that humans are complicated creatures. How we function depends on more than facts and feelings. We also respond to what is left unsaid and we act according to how we imagine that we'll be perceived by others. Even without other people, we still don't exist in a vacuum, inevitably we are influenced by time and space. Like all biological creatures, we have cycles that are naturally sensitive to the time of day.
And, if you step through a doorway, that change in space may for a moment make you forget what you were about to do. Context matters and all of these factors play a part in it: our reasoning, how we feel, the signals we implicitly broadcast between us, our place in the physical world as it changes across time... and yet we can pretend like a lot of these factors are irrelevant when we are online. All the analog content in the world is converted to digital and nobody notices the difference, right? Well, it's complicated indeed.

The digital medium in general and the online venues in particular allow us to negotiate away much of their context in exchange for a lot less friction. Or at least that's the idea, right? You can now attend some conference or a business meeting from your bedroom or kitchen table. So you can give away all the background story and implicit signalling of getting there in exchange for a lot more convenience. However, we can paraphrase Douglas Adams to say that technology is a word for when things don't work yet. If doing all our work online just worked, we wouldn't be talking about "learning the technology" it would just be something that we did. And that could be very valuable for everyone, digital networks are a common good that can empower our education, decrease pollution, give us more time with our families... but they are also merely a tool that can be leveraged to harm everyone, make us dumber, drive us to consume more and pull us away from our loved ones. So, although they can be very convenient, we shouldn't fool ourselves into thinking that one of those conveniences is ignorance. Or going to the extreme of electing politicians that know as little as we do about "the technology". If we don't care about it, there will always be technology, the thing that doesn't work... at least for us. Because the thing does work, but just not for the common good.

People can automate systematic stock trading across continents and yet a teacher still has to drive to the top of a mountain every day in search for a signal to get their smartphone ready for online classes. And the students for those classes also can't find a decent laptop because those are getting harder to repair and are made to become obsolete. The computers most people have access to are closed platforms made for consumers, not citizens. Like the windows of street shops, they are pieces of glass filled with ads and calls-to-action. If we don't have a vision for what "the technology" can do for us, somebody else will take advantage of it. And that is why the clothes don't seem to fit when we reach for something to keep us warm in those virtual realities devoid of context. The internet is no good because your street doesn't seem profitable enough to install infrastructure. Or because the network on our house is just a set of screens that other people want to control to show you ads. Or because arrangements have been made between providers and their prices won't go down to where we can afford them. Or because these companies have come to the conclusion that they don't actually need to hire people to fix things for their customers. Or because we've gotten used to "the technology" just being something that sucks. "The Wi-Fi is down again, I guess. Let's see what's on TV instead..."

Growth of laptop imports in Europe during 2020

As we've seen from the COVID-19 pandemic, "the technology" isn't ready for primetime yet. It's not just the lack of infrastructure, equipment or housing where people can actually have somewhere to work. There is also organizational friction against working remotely. Given the rise of bullshit jobs, as described by David Graeber in his book with the same name, how can those exist in an online world that discards context? These are the jobs that are exactly all about showing up to work. And for as long as we don't claim the public value that advancements in technology can have in terms of giving us less stuff to do and more free time, we are maintaining jobs that no longer need to exist while letting private interests take the initiative towards ending those jobs and capturing the value of automation. Many people are forced to go to work so that the status quo that justifies bullshit jobs can remain unchallenged. Many companies are not willing to spend money to make money by committing to getting employees out of the office and giving them what they need to work from home. Many public services won't improve their processes to a point where they can employ less servants while serving the public better. Overall, remote work forces us to rethink the context on which organisations are built. Without a shared analog space, all that we are left with is that organization's culture as perceived by each person across the internet. Since that's very close to leaving it to speculation, how many organisations are comfortable with that level of individual autonomy? To them, it can almost seem counter-intuitive to have less context and less friction, so of course they act like hitting the breaks when a car starts moving on its own.

As we retreat back to the analog world with few good memories from our online confinement, do we want everything to go back to how things were? Or maybe we caught a glimpse of a level of freedom and power that people would like to have online. Right now, it's an exercise of imagination but no longer science-fiction, it's like we've already been to the moon. We feel like it can be liberating to float detached of context if you're not constantly bumping into obstacles and can control where you're going. Digital networks don't have to be a scary frontier or a distant horizon, they can be public spaces just like a park, a street or a beach. And as much as workplaces extend into our homes, this higher level of democracy and individual power can also extend into our work. Boundaries are being drawn in all these fresh maps of a new digital world, we probably should pick up a pen as well. Any common good can be used against itself if not enough people care for it. So, thank you very much for reading. I hope we all learn more about what we can do and start to have some idea for want we want in the future. Happy new year, everyone.

Do hybrid app frameworks focus on the right problems?

If we could start counting mobile apps on the stores, we would probably see that many of them are, first and foremost, a branding exercise that the current open web cannot satisfy. Institutions want you to have their icon on your pocket and to push notifications into your lock screen. And given how adoption of the internet has been based on ignorance of how it works, people don't know and don't care about the difference between Android, iOS and the web. I mean, users implicitly care about the differences if they affect their experience, but users don't pay for these branded apps, so what they actually care about is just another bullet point that institutions are often unable to support. And, as one might expect, native development costs a lot more than hybrid.

Therefore frameworks like Flutter, Ionic, React Native or even simply Cordova have a real need that justifies their success. But what do we see being offered by them? Do they actually support ignoring the differences between each platform in order to deliver a very similar experience to all of your users regardless of what device they carry? Are they relentlessly on top of possible breaking changes imposed by those who control each ecosystem? Because surely the job of these corporations is not to help hybrid apps thrive, in fact they naturally create problems for them every year with changes in permissions, web views, etc. Hybrid apps are a hacky substitute for the open web, a common good that corporate ecosystems would like to co-opt or make obsolete as soon as possible. Of course they want either exclusive apps or apps that need to generate revenue to support native development from which they can take a cut. If you are none of those, you are merely getting what you paid for your developer's license and that's not a relationship corporations want to maintain on equal footing. They want you to feel like it's a privilege to develop for their ecosystem and not just something you also generate a build for.

However, it seems that hybrid development frameworks prefer to focus on other problems. They even invest a lot into providing you with ways to make your app look different according to each platform, which is the opposite of what brand stakeholders want or even understand. What I might guess is happening here is that everything I said on the previous paragraph is not something you can base any viable project on. You can only focus on what you can control and a real promise of sustainable hybrid development is not something you can commit to. The rug will be pulled from under the next web view or the next permission to access the file system. And when that happens, frameworks cannot just stand there and say "well, it's XYZ's fault", they have to push that problem onto the shoulders of open source. In the end, native problems need to be overcome natively. Meanwhile, hybrid road maps can be filled with achievable things like having buttons with pixel-perfect similarity with those on native apps.

So, as you judge these frameworks, look behind the curtain and see how the issues that handle the usual native issues are evolving. Who is responsible for making sure that next year your app will still be able to access the camera and photos will have the correct orientation? Who is making it easier for you to generate all the graphics platforms require for icons and splash screens? Who is providing feedback on what are the best workflows to get your apps currently published on stores? If you consider these wider questions, some differences between frameworks will still be there, but a lot of them will fade away. Also remember that, if native features are not an issue for your app, maybe it should just be a nice easily-accessible web app.

My particular experience with developing hybrid apps has been very much all about following the needs of event organizers. Events can be an excellent opportunity for web apps to shine. No need to install anything, the event is over after some nearby date and the dynamic nature of its content is well served by a solution that is online first. However, push notifications can be very useful for both attendees and organizers. And that killer feature has driven event apps into the inevitable hybrid territory, at least for a wide majority of smaller brands. It's true that 2020 has been a surprising year for web-based events, but the companion mobile app is not going away. I will probably be writing more about all of them in the future. See you soon.

Thanks for reading! Please subscribe to the RSS feed, maybe follow my Twitter or learn more about me.

Reduce method in JavaScript arrays: useful but maybe hard to review?

One of the best things about JavaScript nowadays is that you can just type "MDN" next to whatever you want to know in your search box and you will get very nice documentation from the Mozilla Developer Network. However, if you go through Array.prototype.reduce(), your initial impression may be "I guess I understand how it works, but what is this good for? Adding numbers, I guess?" So let's take a quick overview on how to reduce an array. Let's say I have a bunch of errors that I want to display in an alert, for example.

let errors = ['File size too big', 'Headers not found', 'Invalid identifier in column B'];
let message = 'Errors found:';
message = errors.reduce(
    (text, error) => text + "\n" + error, 
    message
);
alert(message);

(Quick note about browser compatibility that I think every HTML/CSS/JS article needs to mention: yes, you can use reduce arrays in IE and maybe Android 4.4, no you can't write code in arrow functions for those browsers. However, Microsoft has moved on from IE, so I think we should as well.)

Reduce is a way for you to build one thing out of several things. It's a method you can call on your arrays by giving reduce a function that will iterate through them and a value that can be updated along with that iteration. So, a less debuggable version of this code could be:

alert(['File size too big', 'Headers not found', 'Invalid identifier in column B'].reduce(
    (text, error) => text + "\n" + error, 
    'Errors found:'
));

As I occasionally come back to code that has used reduce, what I think makes it a bit hard to read is the order through which the code seems to flow. You start from the ending, then you go back to check what function parameter carries the value and then you go forward again to see how it gets returned. You can use a named function, but I'm not sure if this seems more readable:

let errors = ['File size too big', 'Headers not found', 'Invalid identifier in column B'];
let message = 'Errors found:';
let reduceErrors = (text, error) => text + "\n" + error;
message = errors.reduce(reduceErrors, message);
alert(message);

For this example, since the returning value is just a string, we could instead use the join method to put together our message...

let errors = ['File size too big', 'Headers not found', 'Invalid identifier in column B'];
let message = "Errors found:\n" + errors.join("\n");
alert(message);

...which seems to be what we actually wanted from the start, right? So, what can justify the added complexity of the reduce method? If, for example, there were errors we didn't want in the message, we could simply errors.filter(error=>error.length<20).join("\n") so reduce is probably more about still including every item while handling each one in possible different ways. So, if we wanted to make the file errors stand out, we could, for example:

let errors = ['File size too big', 'Headers not found', 'Invalid identifier in column B'];
let message = 'Errors found:';
let reduceErrors = (text, error) => text + "\n" + (error.toLowerCase().indexOf('file') > -1 ? error.toUpperCase() : error);
message = errors.reduce(reduceErrors, message);
alert(message);

And perhaps the order in which the information seems to flow can become irrelevant if the function that you pass to reduce can come from anywhere? This theoretically can be any function which takes an A together with some specific B and then returns A according to B. Let's imagine that our example is about loading some Excel file into the browser and that some errors can be found in specific columns. Our array of errors can also result from reducing all those columns into some list of possible issues found in them. And, of course, different functions can be applied to check for these. From these assumptions, we could have some code like:

let errors = [];
errors = files.reduce(checkFileFormat, errors);
errors = columns.reduce(findInvalidHeaders, errors);
errors = columns.reduce(checkIdentifiers, errors);

This looks both more useful and maybe easier to follow. Finally, MDN itself has a lot of useful examples that you probably want to go through, namely for flatenning an array of arrays or for replacing filter and map with simply reduce. It's an invaluable resource for anyone who works with JS and I will probably visit some other topic from them in the future. See you soon.

Thanks for reading! Please subscribe to the RSS feed, maybe follow my Twitter or learn more about me.

Why I use Vim and suck at it

About half a dozen years ago, I was between jobs and had a whole month of August to invest in whatever I wanted to do. So I decided that some of that time should go into learning some coding tool, some skill that could pay-off on the long term. I chose Vim for several reasons, the main one being ergonomics. For me, this is the question of "how is it even possible to consider coding every day for hours?" There are fundamental issues of back and wrist pain that everyone experiences at least once in a while, so finding a good posture that minimizes them seemed a worthwhile investment.

Oh, and when I say "finding a good posture" I already know that "just sit up straight, damn it" doesn't work for me. Does it actually work for anyone? Inevitably, you slouch, forget where your body should be and posture is just something you remember at most once a day. The solution I ended up gravitating towards was just putting my keyboard on my lap. This forces my elbows and shoulders down, which is the easiest way to have a straighter back. I also find that it can be a very comfortable focused posture. The challenge is getting it to work when you also need to use a mouse. And that's where Vim comes in. It's kind of like the same idea of not attacking a problem head-on: instead of "just don't use a mouse", you use Vim knowing that, by the way, you also don't need to move the cursor with your mouse. Much like you type with your keyboard in your lap and now your posture is just better.

However, like many web developers, I don't read and write code 100% of the time. I also go trough many iterations of web interfaces and have to manage a lot of content. Which means I still use a mouse and it's very handy to have a keyboard in my native Portuguese layout. This last point is actually one that turns the Vim learning curve into a bit of a wall. People say "just use :help" but forget that almost immediately you are greeted with something like "Position the cursor on a tag (e.g. |bars|) and hit CTRL-]". And square brackets don't have their own keys in a lot of keyboard layouts, mine requires Alt+8 or Alt+9 to [ or ] for example. Generally, Vim just assumes you are looking at an US/UK layout when you're learning it and, if you don't, you kind of have to go on your own personal voyage of discovery to set it up just as you like it, more than the usual customization that Vim is already all about. I also am forced to use Windows for work, so I don't get the benefit of having Vim inside its real home, Linux.

This does get tiring after a while. There are Vim features that I haven't really gotten around to leverage fully because I'm not sure what's the best way to integrate them with my layout. I learn slowly, but it's not that hard to get going and eventually you do end up with a fun productive tool at your fingertips. Vim is full of unique perks that I also enjoy besides my main reason for using it. It's nice to have a code editor that's the lightest piece of software you need to keep open to do your job. It's just always there when you need it. If the machine is running slow, you know it's not Vim.

And I also find interesting how it's becoming more like a verb than a subject. You can Vim in the command line but also in a GUI. You can also Vim in Visual Studio Code and use the same vimrc configuration with VSCodeVim. You can even Vim in a browser to navigate through pages with something like Vimium. Not to mention that you can Vim into pretty much any text area with VimEverywhere or VimAnywhere. Of course, all of these options have some issues if you want to keep your exact configuration, but the thing is that Vim is here to stay. Both as a real tool and an inspiration for many others. It's simply something you can do if you know how to.

The way that I manage to learn Vim is by picking one feature at a time and trying to incorporate it as best as I can into my workflow. At the beginning, this took about two weeks to get the basics down and be almost as productive as when using Sublime Text, for example. After that, it's all about finding the next snag, saying to yourself "there has to be a better way" and going through the many options Vim can have. Usually, you should start with what you can do just with the current version of Vim and then you can look for plugins that address your problem. For example, autocomplete is a feature that can have many facets and the basic ones come with Vim already, specially if you're coding in older languages. However, you do need plugins if you want full IDE-level auto-completion and code hints. Myself, I've yet to find a plugin that works for me to help with Javascript or PHP development, so I still stick to what Vim already has built-in. And this can include calling out to the command line from inside Vim.

But that's the thing with more advanced plugins in my experience, about 4 out of 10 error out when starting Vim or just don't seem to do anything. They usually depend on other plugins and/or having something like Python working inside Vim. For example, to have even just basic syntax checking, I think I've tried about half-a-dozen plugins across the years and I still end up just calling php -l or tsc to check the file (by the way, for some weird reason, it seems that the Typescript compiler is the easiest command line tool to just check a .js file for syntax errors). Other options I've debated in my head are learning Vimscript to start understanding why are all these plugins blowing up in my face or just switching to VSCode and get baked in IDE features in exchange for a less light-weight experience. Recently, another viable option seems to be using Vim inside the Windows Subsystem for Linux, which might help get some plugins going I guess. Both WSL2 and the new Windows Terminal are still in early days, but there's a considerable momentum for making them viable.

As you can probably gather from all of this, Vim is one of those free awesome tools that you don't pay for with your money but with your time. And about six years using it is not enough, you really have to put in the work to fully optimize your workflow and I actually don't. It's just good enough for me right now, which is marginally better than Sublime Text, but still better where it counts: letting go of the mouse. Yes, it's still all about coding every day and not developing an injury. One other related thing that I'll also mention is having the right keyboard to sit on your lap. I've now settled on a UHK in spite of the problems of getting it to work for me with my native layout (a recurring theme). It's a small programmable keyboard that much-like-Vim is all about doing everything from your home row keys. I will probably be sharing more about my love for mechanical keyboards and Vim in future posts. See you soon.

Thanks for reading! Please subscribe to the RSS feed, maybe follow my Twitter or learn more about me.

How Slack kind of taught me how to make my first bot

Despite having its flaws, real-time chat has become an essential part of my day job. In particular, the mix of features Slack offers was there for us when we needed them. However, it's software aimed at corporations, specially in terms of pricing. Small companies frequently need to involve outside contacts and getting priced per user is a weird reminder that your bill goes up if some friend of the company says "hi". Ironically, the free tier is less stressful and more flexible than having to pay per person. You get to have your small team that has the support of one person that comes by once a month or of another small team with whom you share some channels with. Can you know exactly how many people get to talk this month or the next? Why does that minutia need to be a concern?

So how can a lean start-up pay for Slack? Without any viable option that would allow Slack to take their money, they don't. And so, small teams have to face the limitations of the free tier, namely having only their most recent chat history. Which makes it hard to go back and find this or that important thing someone posted a couple of months ago. But fortunately, at least for now, Slack has an API that allows you to build bots. And therefore, since you can't pay with your money, you can at least pay with your time by just building a bot that does the obvious thing: saving your chat history and having some command to just search through it. It's not a perfect solution because you have to give a search result within a sequence of posts to provide context, but it's still the difference between losing that important e-mail forever or just getting it from your own personal bot.

As this was my first bot project, I've discovered how just saving messages is a reliable way to build a bot that can eventually be more than just a toy. You have a basic bot that can do things with messages and a database. You can probably find out interesting things like what were the most active channels last week or even what projects are taking up the majority of people's time. Furthermore, if you consider alternatives to Slack that also support bots (like Discord for example) and you switch platforms, it's possible to take your chat history with you to be searched through even with the exact same command. On the other hand, you can keep your history for as long as you like and then delete it when you really don't need it, not just arbitrarily losing it when Slack decides.

These chat platforms (including open, self-hosted options like Rocket) do get criticized for bringing nothing new to the table when we've already had IRC for decades. But they have extended the use of bots to what is a more accessible and batteries-included chat experience. And I have discovered that bots can not only be really fun to program, but also surprisingly useful. I will very likely talk about some other experiences I had with these automation tools in future articles. See you soon.

Thanks for reading! Please subscribe to the RSS feed, maybe follow my Twitter or learn more about me.

PHP Upgrade Story: No Data Left Behind

Adjacent problems that weigh you down as you tackle some tech challenge are often as important as that specific tech. I have a PHP upgrade story from version 5 to 7 that illustrates this, showing just another way legacy code can cause issues for years. But it's not really about PHP code, as the codebase was mostly compatible to make the jump from 5.4 to 7.x (eventually 7.3). And when I say mostly compatible, this part of the story is really just about going between the two MySQL extensions, from mysql to mysqli. The rest of the code as far as compatibility is concerned was fine.

To provide some context, this is a somewhat large business codebase that, like any legacy code worth its salt, kept the business running but was difficult to understand, maintain and change. It was the usual LAMP style of old web code with too many global variables and too much PHP mixed with HTML. Gradually restructuring that into something where PHP mostly provided services for single-page applications is not what this story is about, but it was a necessary process to pinpoint exactly what needed to be done.

Given that a considerable amount of important database queries were made out of complicated concatenation of strings, the obvious change that seemed easy to implement (after making sure all inputs were being checked for SQL injection) was to simply go from mysql to mysqli and use the correct encoding for the database. However, the real problem was on the other end. For some reason that I was never able to identify, the text that was being saved to this database by mysql was all garbled up, like "João" instead of "João". This is text that is already in UTF8 getting encoded to UTF8. But this would happen no matter what character encoding set was configured in the database. And testing mysqli on the same database with the same data saved the text as normal, which was good news at least.

So the challenge was how to convert years of garbled up records by mysql into readable text that would be consistent with mysqli. To do this, we experimented with writing several MySQL functions that could helps us identify the fields that had this problem and convert their text to readable characters. We also researched for what would be the best Unicode encoding that would preserve not only the many latin characters we had but also several emoji. We settled on utf8mb4 as tests with several converted pieces of text seemed to show that it could hold all the characters we needed.

Therefore, the basic function we used was something like:

FUNCTION utf8mb4Conversion(x LONGTEXT CHARSET latin1)
  RETURNS longtext CHARSET utf8mb4
  DETERMINISTIC
BEGIN
  DECLARE y LONGTEXT CHARSET utf8mb4;
  SET y = CONVERT(BINARY CONVERT(x USING  latin1) USING utf8mb4);
  RETURN y;
END

And to insure that we were not converting text that didn't fit this latin1 to utf8mb4 assumption, we used something like...

FUNCTION fixOldEncoding(x LONGTEXT CHARSET latin1)
  RETURNS longtext CHARSET utf8mb4
  DETERMINISTIC
BEGIN
  DECLARE y LONGTEXT CHARSET utf8mb4;
  SET y = utf8mb4Conversion(x);
  RETURN IF(latin1mb4Conversion(y)=x, y, x);
END

...in which the latin1Conversion function is just a mirror of utf8mb4Conversion.

So, to recap, we converted every text field in the database to something that humans can read in order to use the mysqli extension, as it works fine while mysql had some weird encoding issue we no longer need to worry about. The main motivation was to unblock the upgrade to PHP, but the immediate benefit was better encoding which allowed us to use emoji characters or easily order queries by their text fields.

I always find interesting how important it is to balance what developers can learn every day from our many resources and communities with the experience we gain from what our particular circumstances demand. In this case, here's a PHP upgrade story that wasn't easy, but it mostly didn't have anything to do with PHP. And this is only part of it, I'll share more in one of my next posts. See you soon.

Thanks for reading! Please subscribe to the RSS feed, maybe follow my Twitter or learn more about me.