Nurding Out, Computer Science

I’ve been working with computers since July 5th, 1976. On that day, I walked into a computer store, the first in the state, and asked for a chance to work with computers.

The owner sat me down at a PolySci z80 computer, loaded BASIC on to it from cassette tape, then told me to have fun with it. He and his partner were having a discussion about me over at the side. I wondered over and asked them how to find the integer value of a number. They gave me the function.

A few minutes later, I had a stream of numbers floating up the screen. I had written my first computer program, a prime number sieve.

This was the first time I had sat down at a computer. It was the first time I had heard of BASIC. It was the first time I had written a computer program.

It worked.

I spent the next few years working with them. Biking, then when I got my license, driving the 5 miles to work every day. I headed off to University to study Computer Science.

The “microcomputers” I learned on were the 8080/z80 and the 6502. When I got to my first Computer Science course, I had to learn to use the punch card machines. Then feed the cards through the reader, wait for the “main frame” to process my job, after which the big printers would spit out my program and results, to be filed for me to pick up.

Whereas the microcomputers could run one program at a time for exactly one person, the owner of the hardware (person controlling the hardware), this main frame could run many programs at the same time. Each program belonged to a different person.

This computer was so powerful, it had multiple CPUs and even had auxiliary processors to help things move faster.

We had one big computer doing the work of many smaller computers. There were no personal computers in the computer science program. When they were finally introduced, they were used as “smart terminals”.

My point in this, is that we made one big computer to do the work of many smaller computers. We gave that one big computer extra as many extras as it could handle.

It had washing machine sized disk drives that held a stack of platters you could take out in a special container. I don’t remember how much they held, but it was likely measured in Megabytes.

When I left to work for the Army, we again had one big computer that many people logged into and did work. The first time I wrote a program on that Cray X/MP I was sure the computer was broken.

After you finish writing your program, you need to compile it. The command to do that is cc test.c Where test.c is the name of the file holding your code.

After the computer compiles your program, it then links your object with system libraries to create a file holding an executable program.

The Cray was so fast that it had finished the compile/link process between the time I pressed “enter” and the time the cursor had scrolled to the next line. I thought it had failed. It hadn’t. It was just that fast.

For the next 15 years, the team I was working with was attempting to create better and better “super computers” by clustering smaller computers together to form distributed parallel processors.

At the same time, we were working with new disk technology, RAID. With RAID, you could combine multiple disk drives to create a larger logical disk drive.

There were three primary technologies involved.

  1. Mirrors. Exactly two drives hold exact copies of each other. These increase read performance and redundancy. A single disk failure does not cause the drive to fail, as the copy is good to go.
  2. Striping. This takes two or more physical drives and treats them as if each drive contained a “stripe” of the logical drive. This could be used to make logical drives that were larger than any physical drive available.
  3. Striping with parity. This takes three or more drives. When a block of data is written, it is written across all but one of the physical drives. The remaining drive holds the parity of each block.

We normally refer to “Striping with Parity” as RAID. In this mode, you can lose any one drive of the array and the remaining drives can be used to reconstruct the missing data.

This means that if you are using a RAID with 7 drives, you are get 6/7 of your disk space available to use.

All of this is very good.

Now we come to the problem, this large drive is attached to exactly one machine. That machine has to share it before other machines can use it, over the network.

You can find these for sale as Network Attached Storage, or NAS.

Mirroring and RAID have a limitation of the drives have to be nearly the same size. If you have 6 1T drives, the last drive has to be a 1T drive as well.

This means you are stuck. If you want to “upgrade” a RAID, you have to create a new RAID with the new drives, transfer all the data to the new drives, then decommission the old drive. Simply put, this is a painful process and not easy. I might have room for 6 drives in the case, I don’t have room for 6 more drives.

The file system which, in my opinion, did the best at this is ZFS. ZFS combined the low level mirroring, striping, and RAID with an actual file system. It was easy to use and had many great characteristics.

Friday, I had 4 ZFS pools of devices on two different computers. Total disk space exceeding 25 TB. The issue was that half of that space wasn’t usable by me. I wanted something that met my requirements of good redundancy and good file sharing and great stability.

I decided to use ceph.

With Ceph, you set up a replication policy. This policy tells ceph how many copies are needed and how distant they need to be from each other.

Consider a ceph cluster with 5 nodes, each node holds 2 Block Storage Devices (osd). If the policy simply has a replication policy of “have 2 copies”. Both copies might be on the same node. If that node fails, you lose your data, until the node is back up.

If you set your policy so that you require 2 copies and those copies are required to be on different nodes, your cluster doesn’t lose access to the data, even if one node goes down.

Ceph is even more powerful than that. It has a hierarchy of groups. Root, datacenter, room, row, pod, PDU, rack, chassis, and host.

A PDU is a power distribution unit. With this level of grouping, you could easily say: I want three copies. I should be able to survive any room in the data center going dark and not lose access to my data.

So where does this put me?

In a stupid place. I had 2 primary home nodes. Those nodes were configured to have one virtual node each, for a total of 4 nodes. This allowed a replication policy of 3 copies. No two on the same OSD. There must be a copy on a different host.

The problem with this, is that I am now burning three times as much space. There are ways with ceph to reduce this, I’ve not done it yet.

Using 3 times as much space meant that I couldn’t move the data I needed to free up the ZFS drives to use else where.

I was recently gifted a computer. That computer now has 4 drives in it. 3 TB, 12 TB, and 12 TB. In 30 minutes, I added 24 TB of space to the cluster, and a new node. I will add two more nodes to make this cluster safe and redundant.

I now have data flowing from ZFS to the new drives. Once that is completed, I will be able to take 12 TB that was part of a ZFS pool and turn them into OSDs.

Life is good, learning is good. Today is rearranging the shop day. I am going to take the CNC router apart, stash the important parts, ditch the things that aren’t worth keeping or turn them into metal for other uses.

This will give me back 1/3 of my shop space, and access to the shelves behind. The new workbench will have a place to go, and I will make progress on organizing my shop. To make it easier to work in.


Comments

7 responses to “Nurding Out, Computer Science”

  1. BobF Avatar

    Same time beginning. I remember upgrading my computer (CPU in the keyboard) to a massive 4K. Yes, K. And that cassette tape audio setting was downright maddening! To this day I hate the asterisk symbol. 🙂

    1. Paul Koning Avatar
      Paul Koning

      What about a computer with a hard drive of 128 kB? No, not a floppy. Those weren’t around yet.

      1. 40+ years ago the GE8800 CT scanner I worked used a Data General Zebra Disk Drive. The drive itself was dishwasher sized and used a removable media consisting of about 10 platters stuck together with a capacity of about 200Mb. Once the disk was full you could either delete data or power down and swap out to a new drive. Each image slice took 9+ seconds to acquire and nearly a minute to perform the algorithmic back projections required to produce one image. A typical head CT was 14-15 images and a typical Abdomen exam was 45+ images. We didn’t do a lot of exams per day back then.

  2. Paul Koning Avatar
    Paul Koning

    I started in 1969, on a 27-bit mainframe, with program entry on paper tape rather than cards — in ALGOL. My father had introduced me to that. He was an ME professor, and used the computer for processing experimental data. He wasn’t a programmer — they had one on the department staff, a geologist by training. (This was before there was a computer science program anywhere in Holland.) But my father discovered that he could read the code, treating ALGOL as “Mangled English”. So he could review the programmer’s work and sometimes find bugs.

    Yes, those washing machine sized drives were probably 80 MB or so. What manufacturer was that mainframe? The description suggests CDC to me. I have lots of good memories of those machines, and still use them (emulated ones).

    1. CDC 750 was the primary with a 6500 as backup. We had developed a shared file system method so that the 6500 and 750 could share one file system. Syncronization was done in shared core memory.

  3. Paul Koning Avatar
    Paul Koning

    On RAID: I like to describe striping (“RAID-0”) as “AID” — because the R means “Redundant” and striping isn’t.

  4. Rob Crawford Avatar
    Rob Crawford

    “For the next 15 years, the team I was working with was attempting to create better and better “super computers” by clustering smaller computers together to form distributed parallel processors.”
    .
    Have you ever looked at the actor model of computation? Think objects, but connected through asynchronous messaging rather than synchronous method calls. It was developed for networks of small CPUs when single-chip CPUs were new, and the modern versions give you network-transparency and all sorts of reliability tools.
    .
    Erlang was the first commercial implementation of the model, and a phone switch built with it was estimated to have 9-9 reliability — 99.9999999% uptime.
    .
    I’d love to be able to use it professionally, and almost had the chance once, but management is more interested in low labor costs than high availability and low hardware usage.