People do a poor job of creating, managing, and remembering passwords. We are horrible at making random numbers and worse at creating things that are random-like but we can remember.

Part of this is because of the rules put in place by NIST and ISO. ISO 27001 has this to say about passwords:

As computers have become faster, the need for better passwords has also increased. Brute forcing a password has a simple cost formula:
$$\frac{{\mathrm{complexity}}^{\mathrm{length}}}{2}$$
For example, if the complexity is all uppercase letters and the length of the password is 8 characters then we have:
$$\frac{{26}^8}{2}=\mathrm{104,413,532,288}$$

Which might look like a large number, but in computer terms isn’t really. As the complexity goes up, the final number goes up. Adding length causes the number to go up even faster. Consider adding the set of numbers, 0-9 to our complexity verse adding one more character to the length of our password.
$$\frac{{36}^8}{2}=\mathrm{1,410,554,953,728}$$
And adding one more character to the length:
$$\frac{{26}^9}{2}=\mathrm{2,714,751,839,488}$$

Adding just one extra character gives us nearly twice as many values to test.

Oh, the divide by 2 is the average number of tests before we guess right.

If the characters are not truly random, the number of guesses decreases substantially. Using names or words, even with character exchanges, produces a much smaller search space. Regardless, the formula stays the same, even if the vocabulary changes.

Consider just using a 3-word passphrase:

$$\frac{{\mathrm{104,334}}^3}{2}=\mathrm{567,868,237,365,852}$$

As you can see, using a passphrase increases the search space incredibly. The only requirement is that the search space of the letter search meet or exceed the search space of the word search.

Unfortunately, many password methods do not handle long passwords well. In early Unix times, no matter how long of a password you entered, only the first 8 characters were used.

Which brings us to

Password Managers

A password manager stores passwords in an encrypted form and retrieves them for you on demand.

For a password manager (PM) to be acceptable to the users, it must interface with the users browsers and other tools that need passwords. This means it must have a mobile app. If it does not, it will not be used.

The PM should monitor applications for password requests and autofill those requests.

The PM must lock itself after a certain amount of idle time or browser/device restart.

Finally, and in some senses, most important, the PM must be secure from data breaches.

To be secure from data breaches, the PM should never store credentials in clear text.

LastPass

This is one of the better-known PMs. While it had a good track record, there was a data breach and credentials were exposed.

One of my clients used LastPass, so I used it. I never particularly liked it. When I could, I moved away from it.

One of the big downsides is that it requires a live, active internet connection to function. No network, no access.

Keeper

I have used Keeper. It is a well-rounded PM with all the expected features. It stores all credentials encrypted by your password. They can’t access your credentials even if they wanted to. Since they can, your passwords cannot be exposed in a data breach.

One of the strong points of Keeper is the ability to share “folders.” You can have a folder for passwords related to a single project or client and share that folder with other users, inside or outside the organization.

The ability to share passwords means that the administrator can update a shared password, and every member with access to that password gets the change immediately.

Shared folders requires a paid tier.

There is also the ability to store small files securely.

The one downside I discovered with Keeper is that it too requires an active internet connection to function.

We were on a long road trip when my kid ran us out of data on my mobile plan. They consumed nearly 10GB of data in a little over 6 hours.

This left me in the position of attempting to log into my provider’s website using credentials stored in Keeper. Except that the amount of bandwidth available to me was so low that it took 30 minutes to get that password and login.

BitWarden

This is my current PM of choice. It provides all the features of Keeper with a few that appeal to me.

First, it is can be self-hosted. This means that all the data security is provided by me. With the self-hosted version, I can offer PM services to anybody at cost to me.

When you move up to any of the paid tiers, the lowest being $4/user per month, you get the ability to create organizations and then share a collection (folder) with that organization.

The mobile application does not need to have Internet access to function, though you might need to request a sync if there are recent changes to your vault.

All data is stored encrypted. The key to decrypt your vault is your master password. Even if there were to be a data breach, your password would still be secure because decrypting your passwords requires your master password.

The BitWarden allows for the use of a Personal Identification Number, or PIN. Unlike most PINs, the BitWarden PIN can be any number of digits. I find that it is easier to remember a number sequence than to remember random character strings.

You can set when the master password is needed to unlock the vault.

If you happen to forget your PIN, you can still unlock your vault with the master password.

Like all good PMs, BitWarden offers two factor authentication (2FA). It supports YubiKeys and TOTP options. TOTP is commonly referred to as an authenticator.

You can use a secondary authenticator for your 2FA to access BitWarden. But you can also use BitWarden’s integrated TOTP generator.

The pricing appears to be reasonable: $4/user per month for “small teams” and $6/user per month for enterprise-level features.

Psono

This is another self-hosted option. It does not seem to have the same polish as BitWarden. It would be my choice if I were just playing.

Conclusion

If you are not using a Password Manager, now is the time to start. For my readers, I’m willing to give you a free account on our BitWarden server, though you are likely better off using BitWarden’s free offering.

My internal infrastructure is getting better and better. Unfortunately, it is still not stable enough.

The router is having issues with memory. I need to add more memory to fix the issues. The problem being that I need to take the router out of production to do so. I’ve not been willing to do that.

The symptom is that connections time out. The fix, restart HAProxy.

HAProxy forwards traffic to the ingress service. This should be running on multiple servers, but it currently is not. There is an issue which I have not resolved where communications from the second ingress service gets lost, leading to the gateway not responding.

This means that when the server that runs the ingress service has to reboot, all ingress stops.

The network is broken into segments, each segment is on a different subnet. Ceph prefers to be on a single subnet.

My solution was to use OpenVSwitch to create a virtual network for Ceph. This works great!

This adds a dependency on OpenVSwitch, which should not be an issue.

The underlaying physical network depends on good routing. The reason I don’t use static is that some nodes have multiple paths and I want there to be multiple paths for every node. This adds a dependency on the routing stack.

Free Range Routing, or FRR, is the solution. It supports OSPF, which is the correct routing protocol for internal routing. It just works.

Unfortunately, FRR and the Linux kernel will stop talking to each other. When this happens, we lose routing of the physical networks.

When we lose routing on the physical network, the OpenVSwitch network stops working.

If the OpenVSwitch network goes down, then the different Ceph nodes can’t talk to each other.

All of this is to say, I’m sorry for the issues you have been seeing with this site. Thank you for hanging in there.

I had to find the sick FRRs and restart them. Once that happened, everything came back to life.

Tuesday I picked Ally up at The Fort at No 4. On the way home, we stopped to do some banking and pick up some groceries.

As we were walking to the bank I said, “Oh, it is closed.”

“Why?”

“Because it’s Sunday.”

“Huh?”

I then got home and posted “Tuesday Tunes” to show up on Wednesday.

I’ve been working on a major website. This includes a content management system for the site that meets the requirements for that vertical.

What I decided on was a “management bar” for those logged in as manager. When they activate the “edit page” function, all the blocks that can be edited are highlighted. Click on the block to start editing the block.

Save your changes. When you are satisfied, click “make live” on the management bar to make all the changes at once.

There are a few good tools for editing text in place.

This left all the other blocks, image editor, carousel editor, calendar editor and a few others.

Design from the top, build from the bottom

I know what I want to do, how I want it to work. Fortunately, I only have to worry about the “working” part. Not the pretty part. I have a team member who helps with that.

What this means is that I see the entire system laid out. This thing will do this, this other thing will do that, the user will see this.

Which leads to a balancing game. How much is done in the template build out? How much is done in the JavaScript module? How much is done on the backend? And how much support does the Frontend request from the Backend?

Currently, I have three different editing models built, each one a bit different from the others. Why? Piece wise progression.

In my original implementation, all logic was done on the Frontend from data provided by the Backend during page load. These led me to a working edit for the carousel. Click here, drag and drop or upload an image. Click there, and you can rotate, mirror, flip, and crop an image, maintaining a fixed aspect ratio.

The next was the text editor. That was simple because the editor works in place, sort of. But it is working. I’ll be adding more features to it, but that is mostly done.

Then the new image processing came into play. Click on the image you want to edit, a dialog pops up, the original, raw image is loaded. Recorded edits are applied, the image can now be edited.

All modals had to be preloaded. All the content of the modal was preloaded. Everything works by modifying existing elements or modifying the DOM. The only communications with the backend are fetching the raw image.

Which led to the calendar editor.

Piecewise progression.

With this, the amount of data started to exceed easy storage in the DOM. Access to needed data was looking more and more like a call back to the backend. The need to serialize objects on the backend for the frontend to manipulate was starting to get stupidly complex.

This led to a redesign. Instead of multiple modals, there is now a single modal (dialog) which is fetched, on need, from the Backend. In this modal, there is a tabbed pane. Click on the tab, a different pane shows.

By listening for a pane to be displayed, we can determine what content we require and request that from the backend, which has full access to all the data and logic required to make this work.

Bingo, everything starts to get easier.

Which means, once this edit is completed, I’ll return to the image editor, make the same design decisions, which in turn will make the carousel editor a simple modification of the image editor.

Things are getting a little easier as I become more comfortable with TypeScript and “promises”

Why the concerns?

First, when I started programming, you didn’t do redundant things because there were no spare cycles and there were no spare bytes.

As an example, I like to write a = do_it(param)->do_other(params2)->do_different(do_wildly_different(param3)).

This seems reasonable to me. No extra cycles, no extra bytes.

Today, it is better to do r1=do_it(param); r2=do_wildly_different(param3); r3=r1.do_other(param2); r4=r3.do_different(r2); This performs the same actions, but it is often clearer to read and allows for checking results at each step. All good.

The other big thing is communications. My last project was a shopping app. Our family still uses it. It creates shopping lists that you can then use from your Android Phone. It has more to it, but that’s the gist.

Because communications is sketch around here, it was designed to work in a standalone state, uploading changes when it could, downloading changes when required.

This lead to an entire mindset of “Communications is expensive”. Which I’ve had to break. The new site makes seriously different design choices.

1. All Manager level actors will have modern browsers
2. All staff working with the site will have reasonable download speeds
3. All volunteers using the site will have reasonable browsers and speeds.
4. All visitors to the site will have a relatively modern browser.

In other words, If you are working on the site, and it takes 5 seconds to get an updated pane or modal, this will be acceptable, or you will need to upgrade your device.

In looking at the current usage of browsers on the Internet, more than 95% of the people using the Internet will do just fine.

Now back to the Bootstrap 5 grind as I design pretty forms.

My formal education is as a computer scientist. I was lucky enough to get this education at one of the cusps of computer development.

My Mentor went through a computer science program just 4 years earlier. There were classes he took that were not even offered in my program. The change was in moving from a need to understand hardware to a place where understanding the hardware wasn’t important to computer science.

What this meant was that my Mentor could design a computer from the gate up, I still struggle with hardware design.

My education included understanding low level instructions, instruction timings, bit manipulation, data structures, semaphores, and a host of other low level concepts. It also included much higher concepts.

At the time, my program included a year-long class where we wrote a working compiler, a required class where we wrote an operating system, as well as all the languages of the day. We even had theory class, such as the class on proving a program correct.

In addition to the formal classes offered by the University, I participated in an intense 8-week course where I was taught how to apply the classroom theory to actual working systems. This was the “systems class”. It started at 0800, ran through to 1200 with no breaks. We had a lunch break from 1200 to 1300. Then classes continued from 1300 to 1700.

We had to turn in our assignment of the day at 0800 the next morning.

This is what a day was like on the third week.

Wake up on a table in the student workroom of the computer center. Collect our work, stumble next door to start our 0800 classes. At 1200 hoof it out of there 2 miles to the house, shower, change clothes, move it back to the computer center and get there before 1300.

Being more than 15 minutes late was failure.

Study and learn new topics from the instructors. At 1700, head out to get dinner. Get back to the computer center by 1830. Work on assignments and projects until the computer was shutdown for nightly maintenance at 0400.

Decide if an hour of travel time to sleep in a real bed was worth more than an extra hour of sleep. Claim a table and fall asleep.

Repeat the next day.

Structured Programming

It is hard for a modern programmer to understand what a huge breakthrough the concept of “if-then-else-end if” was. It was the first of our structured code.

Before we had that type of language structure, we used “if condition goto”. Which was even more confusing when you used the FORTRAN IV construct of “IF value goto1, goto2, goto3” where the goto used was based on if the value was negative, zero, or positive. And yes, there was way too much code that used that instruction.

I helped my father with his MBA by being the computer dude to help him with the software he was using. It was written in FORTRAN IV or III. It wasn’t uncommon to be reading a section of code, have a GOTO that took you somewhere else in the code, which then had another goto back to the place where we just were.

In some cases, the code would conditionally jump to that “patch” which would then jump back into the other routine. It was a mess.


if condition then
do something
else if condition then
do something else
else
do something entirely different.
endif


Structured programming has at its base the concept of block correctness. Every block must be well-defined, do one job, have a set of well-defined inputs and outputs.

If this is satisfied, then you can verify the correctness of that block. You can test the set of acceptable and unacceptable inputs and verify that you get the correct outputs. If this test succeeds, then you can mark the block as ‘good’.

You can combine blocks in sequence. If you are connecting blocks, then the preceding blocks must contain all the outputs that will be used by the following blocks.

You can use conditional structures to create a block that is composed of verified working blocks.

Building from Blocks

One of the things about using blocks, is that you can build iteratively.

To give an example, I am working on a website. The front page has a carousel of rotating “hero” images.

From this, I knew I needed to be able to upload an image. The carousel has a fixed aspect ratio, this meant that I needed to have images in this aspect ratio. I also know that the user will want to decide what part of the uploaded image they wanted to use for the hero image.

In simpler terms, this means that I needed the ability to apply simple cropping to an uploaded image.

There is a black box in my head. It has defined the “cropper” block to take as input an image, the current cropping, and the current canvas to draw on. That block will interact with the user. When the user finishes, the block will output (x, y, width, height) in image pixel coordinates.

There is a different block box that takes two HTML elements and uses them to generate the required inputs to the crop block.

Another block takes the output from the crop block and turns it into a post to the server, to update those values.

Here is the thing, I’m using an obsolete cropping package because it is “simpler” while I’m extending my TypeScript and JavaScript knowledge. But I will be able to replace it with a very modern version and none of the other code will break, as long as the inputs and outputs do not change.

Currently, when you save your changes, the code submits the modifications as form data, which causes the page to reload.

Piece Wise Progression

What this means to me is that I’m constantly testing small changes. I will write a block of code, compile, deploy to the test server, test the results, edit some more.

Saturday was a lazy day. I only performed this cycle 50 or so times.

Every time I get a block working better, I make a git commit.

Friday, I had a breakthrough. I managed to make drag and drop work to select a file for uploading. Created a thumbnail of it. This was all via simple HTML and TypeScript.

Progress was fairly slow on this, learning curve, but what I found interesting is that I would get to a point where I had a working image selection, and only then realize that I had not connected the save button to anything.

Once that was working, the edit process turned out to be more difficult than I expected. It was all working from before, but I needed to hook into the code from a different place. But because that edit process had well-defined inputs, it was merely a matter of creating the correct inputs and it all “just worked”.

Of course, once I click that save button, I found out that I wasn’t actually uploading the image. Which was another thing to fix.

That worked, and it turns out that the server side needed some changes.

But everything was working because the process was all about blocks. When a block didn’t do what I wanted, it was a simple process of checking the inputs for correctness, then checking the output for correctness. If input was good and output was bad, it was a logic error. Track it down and everything gets fixed.

Working On The Railroad

When we code, it is not uncommon to find that there is some exception which can’t be processed by “normal means”.

For example, you are expecting an integer, representing the width of something. You get a string instead.

You could go down a new path, or you can convert the string into an integer. I.e. “768” becomes, simply, 768. That’s simple enough.

But what happens if instead of 768 you get “100%”? The answer could be to go down a separate logic path to deal with width, height, x, and y as percentages.

The railroad method says that you treat the code as a railroad line.

There is the mainline, it is a single track running between two cities. If you have a fast train on the tracks, and it is followed by a slow train, that fast train will get held up behind the slow train.

The answer to this is sidings. Much like the passing lanes on a two lane road, a siding is used to shunt one train out of the way while another train passes.

When the fast training is getting near, the slower train is shunted onto a siding. It waits there until the fast train has passed, then continues down the siding and back onto the mainline.

You can write code this way. When there is a problem with the input, you are being shunted onto that siding. You can decide there is no way to continue and abort, throw an error, you can do something else to get yourself back on the mainline.

Using the “100%” example, the siding means that we will do whatever code is required to convert the 100% into an integer that meats the requirements. This could be as simple as looking up the size of the parent and using that size.

The 100% could mean that we want this size to be a match for the other size. I.e., if the natural size of the image is 640×480, 100% width could mean 640. It could also mean that if we scale the height to 75% of natural height, we want to keep the width as 100% of that natural height.

These logical choices are done, and that conversion takes place on the siding. After the conversion is properly completed, the code can join the mainline again and nothing on the mainline needs to change.

I am actually good at this computer stuff. It is what I do. So I get frustrated when I order hardware and it doesn’t work.

A month ago, I ordered a Mini-ITX board. It was cheap, and it wasn’t 17×17 cm, it was 17x19cm. Which didn’t fit in the case. The low profile CPU cooler/fan I ordered was too tall, so I needed to get a shorter unit. It just wasn’t worth the fight.

I ordered a new Mini-ITX board. This was supposed to be the right size. Not only was it the wrong size, it only had 3 SATA ports, not the 4 on the board I ordered.

That was taken care of with still another motherboard purchase. This time all was good. And amazingly, I didn’t have to wait 2 weeks for the board to get here from China.

Everything is installed, I install the PCI SFP+ NIC, it won’t power up. I don’t know if the card is bad but I replaced the card and that machine is now in production.

Switches are the next big issue. I gave up trying to find a cheap switch, I ordered a new switch, the same as the last I purchased. It should just work.

This switch is an 8 port SFP+ unit, similar to the one in the featured image. It is powered by a 12v DC wall wart.

What arrived was a plastic box with 8 RJ45 connectors. Similar switches can be found on Amazon for under $8. To make it pass the Amazon inspectors, they threw in a free USB-A to USB-C connector. That was returned and a refund was issued.

So I ordered another L3 switch. It arrived in short order. When I powered it up, the power light came on. Then the SYS light started blinking.

Read the manual. Green blinken SYS light means the unit is starting up. Figure that it might be that the unit isn’t fully configured. No problem, I connected it to the network.

Still the blinken light. I scan the network where the device is supposed to be. No response.

Well, the instructions don’t mention the “console” port on the unit (missing from the image because they used the wrong image on the Amazon listing). I ordered a generic console cable to watch the boot sequence, maybe I can spot what went wrong.

What went wrong? Nothing.

The unit is working exactly as it should. I’ve configured it, ready to go into production. The documentation is wrong about what the green blinken light means.

Frustration with hardware, yet this time it is working correctly.

Once this device is in place, every machine, but one, will have at least on 10Gbit fiber link to the main back plain network. Every machine will have a connection to the management network.

As a side effect, it means that each data closet will have at least two fibers into the closet for redundant backup links.

Oh, this also means I can consider using bonded links to get 20G bit transfer rates. That will be cool.

A modern website is a complex system of inter related pieces. Each of which must work correctly and communicate correctly with the other parts of the system.

As website creators, we break the design into the Frontend and the Backend. The front end is anything the end user sees, while the back end is responsible for generating content to be displayed.

The backend is further subdivided into the Model, Controller, and View. The model is our term for the database and database access. The controller is the rules for the website. The view is the creation of content to be displayed.

Consider a website that allows you to purchase an item. The “model” would describe that item. SKU, size, weight, cost, price, images, description, name, and much more. The “controller” encodes the business rules. We can only ship to people that have paid us. And the view turns all the information into content for a browser to display.

The content is delivered as HTML. We can also supply JavaScript code and Cascading Style Sheets. The HTML can have the JavaScript and CSS embedded in the HTML or the HTML can link to other resources to be included with this page.

HyperText Markup Language

The US government wanted a standardized way of creating electronic documents for printing. This was the Standard Generalized Markup Language, ISO8879.

SGML has the advantage of being very application-specific. If you are writing a book, you use one set of tags, if you are creating the Message Of The Day, you use a different set of tags.

The power of markup is that you describe what you are marking up, rather than formatting whatever it might be.

Consider and address. Bilbo Baggins, 999 Bagshot Row, Hobbiton, The Shire. As written in this sentence, it is just a row of text. You could write it that way on a letter and it would be delivered, but the postman would be unhappy at the formatting

<b>Bilbo Baggins</b><br/>
999 Bagshot row<br/>
Hobbiton, The Shire<br/>

Is the address formatted, which looks like

Bilbo Baggins
999 Bagshot row
Hobbiton, The Shire

Using DocBook, a particular version of SGML, we would write that same address something like:

<address><surname>Baggins</surname><givenname>Bilbo</givenname>
<street>99 Bagshot row</street>
<city>Hobbiton</city><state>The Shire</state>
</address>

We do not know how to display the address, but we know that it is an address. If we are provided rules on how to display addresses, we can display this address per the rules.

Structure

HTML was designed to be simpler than SGML. There are fewer tags, and the fixed meaning of the tags made it easy to write HTML by hand.

Almost every post I create is written in raw HTML. That HTML is then styled and displayed in nearly pretty ways.

HTML defined the structure of the document. The structure was of a header section, describing the page, and a body section with the actual content.

Within the content section were the different displayable content. You had headers, levels 1 through 5, you had numbered lists, unnumbed lists, and definition lists (a word with an attached definition). There were paragraphs, links, tables, and finally, there were images.

This content was rendered however the browser wanted to.

There were formatting tags for bold, italics, blinking, and not much more.

If you wanted to “layout” your webpage, you used tables and fought to get things right.

Cascading Style Sheets

CSS allowed us to provide styling to an element. The paragraph above has margins, padding, and boarders applied to it. It has colors applied for the background and for the font. All are set via a style sheet. Your browser has a default style for each element.

The problem that arises is how to attach that styling to particular elements. The answer starts with the structure of the document.

p {
  color: red;
  background-color: green;
  margin-left: 50px;
  border: 2px;
}

This uses a CSS selector, ‘p’ to locate all paragraph elements. It then sets the background to green, the font to red, moves it to the right 50px, then draws a 2px solid border around the paragraph.

This is a basic selector. Selectors get very complex.

DOM

Every element in an HTML document is loaded into the DOM. From there, we can select elements and modify the style of the element with CSS and CSS Selectors.

The simplest method is to give important elements an ID. IDs are unique for a DOM. If there is more than one element with the same ID, this will generate an error, which most people will never see. The rules tell us which element will own that identifier.

To find a particular element with a particular ID you use the ‘#’ symbol. Thus, to find the header just above, we would write “#DOM”. While the header would look like <h3 id=”DOM”>DOM</h3>.

We can add a multiuse identifier, called a class, to multiple elements at the same time. <div class=”quote”> is the code I use to create a quote. The class “quote” has a CSS group attached. This causes all the divs of class quote to be rendered as a block quote.

We then have the tag selector. We used one above with the “p” element. This allows us to select all the elements of a particular type. The selector “li” would select all the list items in the DOM. We could use this to make every line italic.

We can combine selectors to limit which elements are selected. “ul>li” would select all line items of unordered(without numbers) list, while “ol>li” would select all line items which were part of an ordered (with numbers) list.

These selectors can even allow us to move through the DOM in a structured way. We can ask for the first paragraph after a header for special treatment.

DOM Manipulation

When we load JavaScript on a web page, that JavaScript can watch for events on elements. This is done by locating an element with a selector, then watching for a particular event to take place on that element.

The JavaScript can then modify the DOM. This can be by changing the structure of the DOM, or it can be modifying the style of elements in the DOM.

A recent example of this, I added a class to some table data items (td). I did it with a class. I then found all the elements with that class and watched for a mouse click on those elements.

When the click was detected, my JavaScript ran. The JavaScript grabbed the contents of the element, stripped out formatting, then put that content into a text input box, displayed the text input box for the user to edit.

When the user completed their edit, the value they entered was formatted, the input was removed from the DOM. The formatted value was then placed back in the element.

All with a bit of good selection to make it work.

Finally, Selenium uses different types of selectors to find elements for manipulation or testing.

Very powerful stuff.

This is boring geek babble.

Quality Assurance is not something computer nerds are good at. It is boring, repetitive, and difficult.

That doesn’t mean it shouldn’t be done. Instead, it means that you need somebody to do QA for you. You cannot do QA on your own. You won’t see your own errors.

Consider a simple unit test. You have just created a new model (database table). You know you have created it correctly. Some of that is because you trust the infrastructure you are using, but mostly it is because it has worked in the past.

To do a proper unit test, you would need to verify that you can read and write an object of that model. That each functional manipulation does what is should, and that all possible options into the functional unit works.

In the past, I would test student programs that did simple math. For example, they would write a simple four function calculator. I’m the ass that would get their calculator to attempt to divide by zero. They had to handle that case.

The thing that happens, is that as we are developing new code, we test that code, extensively. We know what we are adding and what it should do. We don’t “retest” what we know is already tested and working.

Last Tuesday, that nearly broke me. I had tested my code, was getting ready to deploy the code. Before deploying, I was doing some testing. It wasn’t until I clicked on a link that went to a page I was not going to be testing that I discovered a major error.

I wasn’t even planning on looking at that page.

Here is another example, you have a standardized header for your website. If you check it on one page, why should you test the header on every page? It should work the same. So you don’t test it on every page. Except that there is one page that doesn’t set a context variable, so it causes errors on the page. Because you didn’t test that particular page, the error is missed.

This is where unit tests are a win. In theory, you write a test for every part.

Currently, I’m working with Selinium, This is an API that interfaces to an actual browser. This allows you to control the browser via code.

The basics are you write code to find a page element, you can then verify different aspects of the page.

I’m currently writing code to test the left column. The left-hand column is on almost every page of the website. In the past, I’ve verified the parts of that column I’ve been working on. I haven’t verified the entire column since I first created the site.

Using Selenium, I am able to run the same set of tests against the left column on every page. I can also verify that every menu item opens and closes. I can exercise the entire website.

Because it is so easy to do this, I can just run the tests every time, giving me better results.

Of course there is a learning curve. Of course it takes a different mindset to do these tests. Regardless, it is worth it.

Love it or hate it, project management is a thing. It has to be there. If you don’t think it is there, you are just doing it badly.

Project Managers are a different kettle of fish. Some need to be boiled alive. Others can just dance on hot rocks. And a very few can sit at the big boys’ table.

I’m coming off the end of a rush project that was big. I had to take a customized system and add tariffs to it with about 14 days from concept to deployed. More than a little to get done.

When I started programming, I had a choice of an 8080 with a 24×80 character display, or a 6502 with a 24×40 character display.

When I was introduced to JOVE, Jonathan’s Own Version of EMACS, I fell in love with it. Multiple views into the same file, the ability to copy and paste from different files or different places in the same file. And auto indentation.

Powerful stuff for the time.

My fingers worked will with vi and later vim because I played Nethack and before that, Hack. The programs had a particular key set for moving the cursor based on the key caps of a terminal type used at MIT.

The author had never seen a terminal without arrows over the J, K, H, and L keys. To give you an idea of how ingrained those are, I had to fire up vim and tell my fingers “down”, “up”, “right”, and “left” to record the keys for this sentence. My fingers know, I don’t.

Besides jove, I learned emacs. Emacs is my programming editor. It is what I use when I need to write a lot of code or text. With modern computers, it starts just as fast as jove ever did on a 68020 class CPU.

The problem we had was keeping track of what needed to be done or fixed. This might start off as a document, written with jove in troff. This could be fed to different processors to create PostScript files to be sent to our printers.

Later, some of us used LaTeX for the same thing. Your “design document” was a separate file that was “fixed” before you started coding. These documents never contained more than brief pseudocode and discussions of algorithms.

As you were coding, if you discovered something, you created a comment and marked it. The two most common marks were, XXX which meant that the code was broken in some way, but it didn’t need to be fixed now. All XXX marks had to be addressed before the code could be released.

The other mark was TODO. This was working code but needed some features or extensions added. These did not need to be fixed before release.

In general, we used grep to find all these markers in a list of files. It wasn’t difficult.

The small program I’m working with has some 250k lines of code. After 3 or 4 years of supporting this site, I would say I’ve looked at every line of code in the system.

Finding every marker in 4100 files across 1200 directories is a pain.

Enter Kanban

Kanban is a project management tool. The concept is easy enough to do with sticky notes and a white board or notes with push pins on a larger bulletin board.

Today, the normal Kanban has 4 columns to hold cards. The cards are labeled, “backlog”, “To Do”, “Doing” or “Working”, and “Done”.

When you create a card it goes into the “backlog” column. These are issues or tasks that have no resources assigned to them.

Once per week, there is a meeting of the workers and the project manager. In this meeting, the project manager evaluates the cards that are in the “Done” column. If they are truly done, then they are removed from the board and added to the QA project.

Cards that are in the working column stay in the working column. Cards that are in the working column can be moved into the backlog column if some other card blocks them.

For example, if you have a card that says, “Put new tire on left front wheel” it cannot be worked on until the card that says, “Purchase a new tire for the front left wheel.” Until the purchase card is completed, you can’t work on the installation card.

If there are any resources (workers/developers) that think they are going to need more tasks to work on, the project manager will take cards from the backlog column and move them to the To-Do column.

When a worker requires more work, they move the card from the To-Do column to the working column. When they complete the card, they move it to the Done column.

I’ve used Kanban in the past. It never really appealed to me as it didn’t feel any different from the old ways of doing things.

For this latest project, I used my Kanban board.

Instead of putting markers in the code, I opened a new issue. That issue just went into the “backlog” column. I could tag the issue as a bug or a feature. I could indicate that cards were blocked. It was faster to create the issues/cards than to make entries into the files and then try to locate them later.

Today, I’ll be looking through anything in the QA column and writing unit or web tests for them. I’ll also be doing a QA across the site, to add to the project board.

The biggest thing for me was the ability to visual see what still needed to be done.

Conclusion

Good tools make the work go faster.

In computer languages, there are very few that are structurally different.

FORTRAN is like COBOL, which is like Pascal, which is like BASIC, which is like ADA, which is like …

Forth is not like those above. Nor is APL or Lisp.

Assembly languages can be used in structured ways, just like FORTRAN, COBOL, Pascal, and many others. It requires the discipline to use if not condition jump skip_label; do stuff in condition; skip_label:. The actual programming logic stays the same.

The two computer languages I dislike the most are PHP and Python. Both because they are weakly typed.

In a strongly typed language, you declare a variable’s type before you use it. The type of the variable is immutable for its lifetime.

In other words, if you declare a variable of being of type integer and then attempt to assign a string to it, it will barf on you during compilation.

In PHP, all variables look the same, any variable can hold any type at any moment. The type can change from line to line. And the language will do implicit type casting. It is hateful.

Python has all the same characteristics I hate in PHP, with the added hateful feature of using indentation instead of begin-and markers for blocks.

I’m lucky that Python has an optional typing capability, which I use consistently. The optional part is a pain when I want to use a module that has no typing information. When that happens, I need to create my own typing stub.

But the worse part of all of this is that they get jumbled together in my head. How many entries in an array? In PHP, that is determined by the count() function, in Python it is the len() function.

In Python, the dot (.) is used to access methods and attributes of objects. In PHP, it is the concatenation symbol.

I am tired of writing Python in my PHP files and I dread switching back to Python because I know my fingers will mess things up.