Skills

My mentor, Mike, use to say “There is always a bottleneck.”

What he meant by this, was that for any system, there will be a place which limits the throughput. If you can find, and eliminate, that bottleneck, then you can improve the performance of the system. Which will then slam into the next bottleneck.

Consider this in light of traffic. It is obvious to everybody, because it happens every day, that traffic does a massive slowdown just past the traffic signal where the road goes from four lanes to two. That is the point which we want to optimize.

The state comes out, evaluates just how bad the bottleneck is. The money people argue, and 15 years later they widen the road.

They widen the road between the first and second signal. Traffic now clears the first traffic signal with no issues.

And the backup is now just past the second signal, where the road narrows again.

We didn’t “solve” the bottleneck, we just moved it.

With computers, there are many bottlenecks that are kept in balance. How fast can we move data to and from the network, how fast can we move data to and from mass storage, how fast can we move data from memory? These all balance.

As a concrete example, the speed of memory is not fixed at the speed of the socket. If there are more memory lanes or wider memory lanes, you can move data faster.

If you have a fast CPU, but it is waiting for data from memory, it doesn’t matter. The CPU has to be balanced against the memory speed.

My mentor was at a major manufacturer, getting a tour and an introduction to their newest machine. He had an actual application that could also be used for benchmarking. One of the reasons it was a powerful benchmarking tool, was that it was “embarrassingly parallel”.

In other words, if it had access to 2 CPUs, it would use them both and the process would run twice as fast. 8 CPUs? 8 times as fast. Since the organization he worked for purchased many big computers (two Crays), and he was the go-to guy for evaluating computers, his opinion meant something.

He ran his code on a two CPU version, found it adequate. Requested to look at the actual designs for the machines. He spent an hour or two pouring over the design documents and then said.

“We want an 8 CPU version of this. That will match the compute (CPU) power to the memory bandwidth.”

The company wasn’t interested until they understood that the customer would pay for these custom machines.

Six months later, these 8 custom machines were in the QA bay being tested when another customer came by and inquired about them.

When they were told they were custom-builds, they pulled rank and took all 8 of them and ordered “many” more.

What happened, was that my mentor was able to identify the bottleneck. Having identified it, he removed that bottleneck by adding more CPUs. The new bottleneck was no longer the lack of compute power, it was memory access speed.

The Tight Wire Balancing Act

I deal with systems of systems. It is one of the things that I was trained in. I.e., actual classes and instruction.

Most people have no idea of how complex a modern Internet service is. I.e., a website.

This site is relatively simple. It consists of a pair of load balancers sitting in front of an ingress server. The ingress server runs in a replicated container on a clustered set of container servers. The application has a web service provider that handles assets and delegates execution to an execution engine.

This runs a framework (WordPress) under PHP. On top of that is layered my custom code.

The Framework needs access to a database engine. That engine could be unique to just this project, but that is a waste of resources and does not allow for replication. So the DB Engine is a separate system.

The DB could run as a cluster, but that would slow it down and adds a level of complexity that I’m not interested in supporting.

The DB is then replicated to two slaves with constant monitoring. If the Master database engine goes offline, the monitors promote one of the slaves to be the new master. It then isolates the old master so it does not think it is the master anymore.

In addition, then non promoted slave is pointed at the new master to replicate.

I wish it was that simple, but the monitors also need to reconfigure the load balancers to direct database traffic to the new master.

And all of this must be transparent to the website.

One of the issues I have been having recently, is that in the process of making the systems more reliable, I’ve been breaking them. It sounds stupid, but it happens.

So one of the balancing acts, is balancing redundancy against complexity, against security.

As another example, my network is physically secured. I am examining the option of running all my OVN tunnels over IPsec. This would encrypt all traffic. This adds a CPU load. How much will IPsec “cost” on a 10 Gigabit connection.

Should my database engines be using SSD or rust? Should it be using a shared filesystem, allowing the engine to move to different servers/nodes?

It is all a balancing act.

And every decision moves the bottlenecks.

Some bottlenecks are hard to spot. Is it a slow disk or is it slow SATA links or is it slow network speed?

Is it the number of disks? Would it be faster to have 3 8TB drives or 2 12TB drives? Or maybe 4 6TB drives? Any more than 4 and there can be issues.

Are we CPU bound or memory bound? Will we get a speedup if we add more memory?

Conclusion

I ave so many bottles in the air I can’t count them all. It requires some hard thinking to get all the infrastructure “right”

There’s a difference between thriving and surviving. I notice a trend among those who call themselves preppers. There are two sorts: those who are prepared to simply go on living, and those who prepare to survive the apocalypse. The first sort have plans for growing food, hunting, trapping, collecting water, providing continuing light, creating electricity in small amounts, and a lot more. The second sort treat it like it’s a weekend warrior camp where they get to play commando.

I am among the first group. I hone my skills so that I can thrive, rather than survive. Yes, survival is important in the early days of any disaster. You have to make it through the mud slides or tornado or tsunami or whatever it is that went boom. The bottom line is, surviving is the easy part. Either you survive, or you don’t, and the answer to that question is going to come up pretty quickly. No matter how much you prep, there’s always going to be some kind of emergency that you didn’t think of that could wipe you out. Thriving, though… or, as I like to call it, “living,” is what it’s all about.

After whatever immediate circumstances mess up our civilization (because let’s be serious, the Earth, our world, is going to go right on spinning in space… unless the emergency has to do with Vogon Constructor Fleets, but that’s another matter entirely), we’re going to want to focus on getting things built into some kind of new normal. I’m one of the type of folks who hope the new normal isn’t like the old normal. I’d like to see less government, and more growing of food. But that’s me.

How do you prepare to thrive? Well, you get your life together in such a way that a disaster is only a momentary blip. I’ve chosen to do this by studying how medieval and early Colonial people survived in some of the most terrifying circumstances they’d ever encountered. I don’t need to re-invent the wheel, after all. I can see clearly how they did it. And then I practice those skills, over and over again.

Read More

Monday, I had an opportunity to visit the SIG Academy/SIG Experience Center.

In the late 70s, I had a chance to visit NYC for the first time. That feeling of awe, looking up at the skyscrapers. Trying hard not to have pidgin droppings fall into our open mouths.

That is sort of how I felt walking into the building. I spent a long time in the museum portion of the building. I was surprised at the lack of firearms from the 1700 and 1800 hundreds. Starting in the 1900s, they had a presence.

One of the people who worked there was willing to discuss the things that are coming out of SIG for the military. One of the coolest is their short stroke piston operated rifles. Using a new caliber, they are getting good velocity out of shorter barrels.

I want one of those belt feed rifles. They might be out of my price range.

Part of the coolness factor is that with the dual action bars with the short stroke piston, they don’t need buffer tubes. This allows for true folding stocks. Or, something that was just FUD sick.

They took this beautiful action and shoved it into a plastic “hunting” rifle. No pistol grip. No buffer tube. It doesn’t look like an AR platform in any way, unless you shove a 30 round magazine into it.

I’m hoping for a version is 7.62×521(Win .308). That would be a nice rifle. No scaring the mundanes, packs a punch, light weight and reliable.

Unfortunately, I got to looking at the display case full of pistols…

Wouldn’t you know it, a cute little black guy followed me home.

Now, I’m a firm believer in my 1911s. I love the feel of them. I love shooting them. They are tack drivers.

I think I’ve found a new love. The P365 x macro.

This guy fits my hand perfectly. It doesn’t point exactly like the 1911s, but close enough. The grip size is perfect, if it wasn’t, you just replace the back strap. The gun comes with three different back straps.

The one I took home has an external safety, this is to standardize my manual of arms.

On Tuesday, I went to the range and put rounds down range. FUN!!!

I have three plates set up. 1/4 torso behind a round gong and a 1/2 torso to the side. One of my drills is to hit the head of the target hiding behind the gong, then hitting the 1/2 torso to the side, then back again.

With 17 rounds in the magazine, the grip wasn’t double stack wide. It performed admirably. From first to last round, it was consistently ringing steel.

The only downside is the magazines. You will want to use the loading tool to help load the magazine. Even with the tool, getting rounds 14 through 17 into the magazine was a pain. In some ways, it reminds me of loading the M3 grease gun magazines. Heavy springs to push those rounds reliably all the way.

The other thing is that I don’t like the bright orange followers in the magazines. I haven’t looked, but I’m pretty sure I can find replacement followers.

Now for the next bit of coolness, this thing has a drop in FCU. It is the FCU that is the registered firearm. This means that you can pay once for the FCU, then have multiple frames that you can put the FCU into.

Want a sub compact? Buy the frame, barrel, and magazines, you are good to go.

Want a full size? Buy the frame, barrel, and (maybe?) magazines, you are good to go.

I am going to add more SIGs to my collection.

Two is one, one is none. Have more.

It’s that time of year. Most of us enjoy a turkey over the Thanksgiving holiday weekend. My family tends to do the “friendsgiving” thing on Saturday, but we also have a small gathering on Thursday afternoon. Turkey is ubiquitous. It’s also a bear to cook, if you ask many chefs. I’ve never understood that. My turkey always turns out moist, delicious, falling off the bone, and perfect. Maybe I’m just special? Never mind, I’m going to share my turkey secrets with you, so that you can also have a perfect turkey this Thanksgiving!

Ingredients:

• one turkey, 15-22 lbs
• stuffing of choice
• Bell’s seasoning
• bacon or butter
• fresh herbs (parsley, sage, rosemary, and thyme are popular)

I prefer to make my turkey in a Westinghouse, because it frees up my oven for pies and other sides. The cooking method is exactly the same, though, no matter what container you use.

To prepare your turkey, make sure that all the giblets are out of it. Check the main cavity, but also the space around the neck. Sometimes, butchers like to hide little gifts in there, and it’s unpleasant to realize that half way through the cooking process, when the plastic they’re wrapped in begins to melt. I like to rinse my turkey, inside and out, but that’s a personal choice. If you do rinse it, make sure that your sink area is clear of all items that might be besmirched, and do a bleach wipe afterward. Turkeys (and chickens) can carry salmonella and other stuff, and even a little of it lingering on a countertop is bad. Use a lint free towel to dry off the exterior of your turkey. Salt the inside of your turkey well by taking a handful of salt and rubbing it on all the interior surfaces. This isn’t meant to be caking it on. Think of it as like a dry rub for steaks.

Add your stuffing. My family uses a variety of recipes depending on the year. We have a wild rice and sausage meat stuffing, and a more traditional bread and liver stuffing (nicknamed “heart attack stuffing” because of the “one egg per pound of turkey” rule used in it). If you don’t feel like making a stuffing, add the heel of a loaf of bread and one quartered onion to your turkey’s cavity. You can also add an apple or orange, if you like. Do not leave the cavity empty, as it changes how the turkey cooks. If you are in a rush, feel free to make up some “stove top stuffing” and jam it up inside there. I’m not going to judge you. Put your raw, stuffed turkey into the roaster pan or Westinghouse and tie the legs together with the tail tucked up in between them. Tuck the wings down into the bird’s underarms as best you can. Don’t panic if it doesn’t work; if the wings are a bit crispy at the end, they’ll be perfect for making turkey broth the day after Thanksgiving.

Using your hands, loosen the skin over the breasts of the turkey. There are several ways you can proceed here. My personal method is to stuff part of my bread stuffing up into the breast area, where it will keep the breast meat moist and help it cook. However, if you’re not using a bread stuffing, you can instead rub the breast meat UNDER the skin with either bacon or butter. Leave little bits of it under the skin. Slide your fresh herbs up under the skin, too, if you like. I almost always slide in a sprig of rosemary and a few leaves of sage from my garden. These will flavor the breast meat, especially as that bacon or butter melts into it.

Sprinkle the outer skin of your turkey with Bell’s seasoning and a bit of salt, pepper, and paprika (REAL paprika, not the red sawdust that passes for “paprika” in most stores). The skin should be evenly speckled with the seasoning, not covered entirely. Make sure to get some of that on the wings and legs, as well as the breast. Cover the turkey with a lid or tin foil, and put it into the cold oven or Westinghouse.

Bring your oven or cooker to 450F. Add a bit of olive oil or bacon fat to the bottom of your turkey roaster or Westinghouse pan, to help keep your turkey from sticking. If you want it to be really rich, you can melt a stick of butter in there, but beware that it may smoke due to the high temperature. Once the oven is to temperature, let it sizzle for 15 minutes, no more. Turn the heat down to 300F, and walk away.

You will not look at your turkey again until an hour before you think it’ll be ready. I mean, you can peek now and again, but you don’t need to baste or anything else. I will sometimes suck the fat out of the bottom of the roaster, if I think it’s getting too much, but that’s an individual thing.

Turkey cooks for about 20 minutes per pound of turkey, when stuffed. That means if your turkey is 21 lbs, like mine, it’ll take about 7 hours (yes, SEVEN) to cook (20 minutes per pound, that’s 3 pounds per hour, divided into 21 lbs, which gives you an answer of 7). When you get to the bigger end of turkeys (over 18 lbs), the timing gets wiggly. I might only have my turkey in for six hours, so I have to watch it during the last couple of hours to make sure it isn’t overcooked. For anything under 18 lbs, though, you’re safe to count 20 minutes per pound of turkey. Always start checking on it an hour before you think it’ll be ready, though.

Having a meat thermometer on hand is really handy. I use mine all the time. Turkey must be between 160 and 165F internally to be properly cooked. Turkey also continues to cook and rise in temperature for a full 20 minutes after it comes out of the oven. I always aim for 160F, knowing it’ll go up more as it rests before I slice it up.

If you don’t have a meat thermometer, or don’t care to use one, you can also do the “leg wiggle” method of testing. As you near the end of your estimated cooking time, wiggle one of the legs gently. You may need to pierce the skin if there’s a lot of liquid inside, which is fine. Your turkey leg should be very loose, almost coming off, when it’s ready. If your turkey leg comes off in your hand when you go to wiggle it, it’s definitely ready. Don’t panic if it does… this method of cooking will result in it not being dry even if you’re a little over in your cooking time.

Turkey MUST rest for a minimum of 15 minutes before you cut into it. Take it out of the oven, and out of the roaster. Place it on a large cutting board. I usually place a handful of towels I don’t much care about underneath the board, because the juices will run when you start cutting, and it’s easier to clean up if it’s all just drained into a towel. The resting time allows much of the liquid to be reabsorbed into the turkey meat itself, making it taste incredibly moist and delicious. It also lets it finish cooking. While you’re waiting, make some gravy with the pan drippings!

Don’t forget to take your stuffing out before you carve the turkey. You can do that at the 15 minute mark, if you like.

I usually cut the turkey up onto two platters: one dark, and one light meat. The breast is the most difficult part to cut up, in my opinion. It’s always at an odd angle. Take a few pictures of the turkey before you start cutting it up, and then carefully (using silicon heat pads if necessary) remove the two breasts and put them flat on the cutting board. It should be fairly easy, by cutting along the breast bone and then just gently pulling downward. Once on the cutting board, your turkey breast meat will slice up quickly and neatly. After that, you can pull off the legs, thighs, wings, and whatever dark meat you can get at. I can’t stress how useful silicon gloves are, because you can clean them in a way that fabric ones can’t be, which means you can grasp the various parts of the turkey more easily.

I know it seems like a lot of work. It really isn’t. Most of my turkey time is spent doing all the other assorted foods that go along with turkey. We usually eat around 5pm or so, which means I’m not getting up at the crack of dawn to try and do a gyn exam on my turkey before I’ve had coffee. I can get up at a reasonable hour, and set it all in motion by 10am. Once you’ve done this a time or two, it’ll become second nature. It’s such an easy way to prep the turkey.

A few notes:

• Don’t pre-stuff your turkey the night before. You can make the stuffing ahead of time and have it ready to go, but letting it sit overnight inside the turkey will be a sure way of getting everyone to the ER in record time. Don’t do it.
• Do invest in silicone gloves, a decent carving knife (NOT an electric knife), and the Bell’s seasoning.
• If you use bread stuffing, REALLY stuff it in there. Cram it in. Every chef I’ve watched on television goes on and on about how you should never over stuff your turkey For Reasons. Ignore them. They’re wrong. Over stuffing your turkey will result in the turkey disjointing itself during the roasting process. This makes carving it up later SO much easier. There’s something special about opening the turkey roaster and just looking at your disjointed turkey, held together only by skin.
• Ignore those stupid pop up timer things that come in some of the turkeys. They often don’t work, and sometimes they pop early. Just pull them out and toss them.
• Remember to defrost your turkey, folks. None of this will work with a frozen turkey. Don’t try and cook a turkey from frozen, please. Even if you get one that says it CAN be cooked from frozen, just don’t. Whole turkeys should be slow roasted to preserve the moistness and flavor. Here’s a link to a handy defrosting schedule.

I’ve had a slow leak in the right front wheel of my truck for the last year. When I went to have it fixed, I found that it wasn’t the tire. It was the wheel that was leaking.

Leaking tire? $20. Leaking wheel? $150+

Well, that slow leak isn’t slow anymore. Ally asked for a ride to the store. When we got to the truck, the left front tire was flat.

I could have pulled the air hose to the truck, filled the tire, moved the care to the garage and changed the tire there. But…

I decided that I was feeling old, so I was going to change the tire using only the in vehicle tools.

The tools were not where I expected to find them. Which makes sense. I looked for them under the left rear passenger seat. It was empty.

You don’t put the tools on the left. That would have the driver’s door open for an extended period of time with the driver not paying attention to traffic. Of course, it is under the right rear passenger seat.

The package consists of a hook on a rod, 2 straight rods, the tire iron, and a column jack.

I put the hook and extensions together, then went to find the magic hole to lead to the tire let down. Do you know it is designed to be done in the dark, with your eyes closed? There is a cup designed to accept the hook end. Just feed the rod in until it can go any further, rotate counterclockwise.

This old man was under the truck looking. Trying to find that stupid hole to feed in the rod. I got it done. I got the tire out.

Then it was time to do the tire change. A trick for youngsters, break the nuts free before you lift the tire off the ground. Much easier when the tire doesn’t spin.

In the course of breaking the lug nuts free, I manged to break a stud. Not a huge issue. Just another one of those things.

In the end, I used the car jack to get the tire off the ground. It wasn’t difficult. Surprisingly so. I used the tire iron to break the lug nuts free. After the one stud broke, I did as well.

I finished using a 1/2″ breaker bar and a floor jack.

In the end, I felt like I was able to do the task with the tools at hand.

Practice Your Skills, Even the simple ones

Is this something I could have done faster and easier with my impact wrench, floor jack and such? Yes. But I would not have practiced a skill I want to have. Practice, practice, then practice some more.

Story Time

Years ago, mom took the VW Micro bus in to the tire shop to get new tires put on. She was instructed to tell the staff that she wanted the lug nuts torqued to a specific setting. Per the manual.

When they were done changing the tires, she asked, “Did you torque them as specified?” To which the manager replied, “Of course we did.”

Mom got the bus and drove it around to the entrance. Took the tire iron from the roadside tool kit and went back inside.

She told the manager to come take on lug nut off and replace it on each wheel. The manager shrugged and called one of his guys to take the car back into the bays to do what was requested.

“No, you do it with this”, holding up the tire iron.

He tried, was unable to do so. Mom explained that she needed to be able to change the tire if she was alone with her kids. That she wasn’t going to be able to call him to do it for her when on a road trip to Wisconsin.

The manager “got it”. He sent the bus back in and they torqued the lug nuts to spec.

So there’s an emergency. The power is out, and your neighbor comes by to borrow a candle. It’s no big deal, you lend them one of your hurricane lanterns, so they can have light and be safe. You don’t even think twice about it. When it’s a short-term emergency, this is a standard response, and it’s very reasonable. When there is a better-than-coin-toss likelihood of replenishing your stash of any given item in the near future (“near” being defined however you like), lending or gifting is not a big deal.

If we’re dealing with a big emergency, though, this may not be the case. Suddenly, giving something to your friend or neighbor seems a lot less reasonable. You have to weigh the possibility that you may not be able to get more of whatever it is you’re lending, before you run out yourself.

It’s not easy to say no. It’s a skill, and it’s one you have to practice, as with all other skills. Unlike most of the other prepping skills, it comes with an emotional cost that far surpasses its use.

Consider this scenario: It’s the apocalypse, however you see that. You’re fairly well situated, and have “enough” of stuff that you’re not hurting. But you’re budgeting every calorie, and watching the weather to know what to do next. Your neighbor comes by and asks to “borrow” a candle. They have kids, and those kids are afraid of the dark, and she knows you have extras… Suddenly, it’s not so easy to say no. Note the kids, because that’s the gotcha that many people will inveigle into the conversation if they think it’ll get you to part with your goods.

It’s easy to say that they ought to have prepared in advance, like you did. And it’s true, the parents should have prepared. The kids, though, are not responsible for their parents’ stupidity or lack of forethought. So what do you do? You still say no.

Read More

I’m not a huge creamed corn fan, but wow, this was delicious! It was a bright, sunny looking meal on a miserable, chill evening. The sweetness of the corn complimented the savory chicken, and the entire dish came together in under an hour. I hope you enjoy!

Ingredients:

• 2 lbs chicken breasts, cut into strips
• 1 tsp onion powder
• 1/2 tsp garlic powder
• 2 cans of corn (15 oz each) strained
• 1-1/2 cups milk
• 3 tbsp olive oil
• 1 large onion, diced
• 4 cloves of garlic, minced
• 1 tsp of dry oregano
• 2 sprigs of fresh thyme
• 1 pinch of red pepper flakes (optional)
• 2 tbsp butter
• 1/3 cup Parmesan cheese, freshly grated
• salt and pepper to taste
• fresh cilantro for garnish (optional)

On a plate or platter, lay out the chicken strips. Drizzle them with a tablespoon of olive oil, and season with salt and pepper, onion powder, and garlic powder. Set the chicken to the side.

Add the first can of drained corn into your blender or food processor and add in the milk. Blend until this is smooth, then set it aside.

In a large sauce pan, heat the remaining olive oil over a medium high heat. Add in the chicken strips and sear until they are golden brown. Don’t rush this! You  may need to do it in stages, depending on the size and depth of your pan. Don’t crowd the pan; it’s better to do several batches than to try and shove them all in at the same time. When the strips are seared, remove them and rest them on a plate.

In the pan you just removed the chicken from, add in the onion and saute until it’s soft and translucent. Add in the minced garlic, and saute until it is fragrant, about a minute. Add in the pinch of red pepper flakes (if desired), oregano, and thyme. Stir to combine.

Pour the corn and the corn and milk mixture into the pan over the onions and herbs. Stir well, and then simmer until it begins to thicken. Season it with salt and pepper, to taste.

Remove the sprigs of thyme, and add in the butter and cheese. Fold it in gently, and let it simmer for another few minutes until it’s all incorporated. Return the chicken and the juices to the pan, cover, and let it simmer for another few minutes until the chicken is cooked through and the sauce is thick and gravy-like.

Notes:

I didn’t use fresh thyme. Instead, I used a teaspoon of dried thyme. It worked fine! I used oat milk, because I can’t do dairy, but regular milk would be great. I used plant based “butter,” and I added the cheese at the very end, so I could have mine (with plant based “cheese”) separate from everyone else’s (with real cheese). That worked out relatively well, to be honest. I served it up with some pasta, but it would have gone equally well with rice or mashed potatoes.

It does need a vegetable, as corn really doesn’t pause much on its way through you. I went with carrots, because it was such a bright yellow that I thought it needed some orange in there. I boiled them, then sauteed them in olive oil and garlic, and served them on the side.

Networking used to be simple. It is unclear to me why I think that. Maybe because when I started all of this, it was simple.

Networks are broken down into two major classes, Point-to-Point (P2P) or broadcast. When you transmit on a P2P port, the data goes to a dedicated port on the other side of a physical link. There it comes out.

Each port is provided an IP address. A routing table tells the router which port to transmit on to reach a particular network. A router works in a store and forward procedure. It reads the entire packet from a port, then retransmits that packet, modified as needed, on a different port.

A broadcast network is one where multiple devices are connected to a single physical network. What is transmitted on the link is heard by all the other nodes on the same physical network.

Originally, that physical network was a switch. Your network card would connect to a switch, the switch then transmits everything it receives on one port to all other ports.

Switches could be connected to each other. The only requirement was that of time. The amount of time it takes for a packet to travel from one end of the physical network to the other was limited. If it took more time than that limit, the network became unstable.

This concept of everything going back to a single switch was expensive. The cabling was expensive, the switch was expensive, the network card was expensive. A working network started at around $50,000. $30K for the switch, $10K for each network card. Hundreds of dollars for cabling.

The original Internet protocol was only going to have addressing for 65,000 machines. How many machines would be network attached if each site required $50k just to get one or two machines hooked up. We compromised at 4 billion.

We are working on getting everything on IP version 6 with 18,446,744,073,709,551,616 IP addresses. I think somebody told me that that is enough addresses for every atom in the known universe to have an IPv6 address.

From those expensive switches, we moved to 2-base-10 and “thick” Ethernet. These had the same limitations, but the costs were starting to come down. Something around $1000 to get into thick net and a few hundred to get into thin net.

Routers were still expensive. With the advent of 10baseT, we saw costs drop again. You could get an Ethernet hub for under a hundred dollars. Routers were only a few thousand. The world was good.

The other day I purchased an 8 port 10 Gigabit router for under a hundred dollars. It has 160 Gigabit internal switching. This means it can move 10 Gigabit per second from and to every port.

It cost less than $35 for two fiber transceivers. It cost around $33 for an Intel-based NIC capable of 10 Gigabits.

This means that I can upgrade a server to 10 Gibibit capability for around $60. Not bad.

A Step Forward

My data center was rather small. It was set up as a single /23 (512 addresses) connected via L2 switches. The switches were all one Gigabit copper.

You can buy 10 Gigabit L2 switches, but they are either copper, with limited distances and a need for high-quality cabling, or they are expensive.

Moving to an L3 device got me a better price and more features.

Moving to an L3 router gave me some more options. One of the big ones is the ability to have multiple paths to each device to provide high availability.

This requires that each node have multiple network interfaces and multiple routers and switchers. With the routers being cross connected, with each node being able to handle multi-path communications.

This is the step forward.

A step backwards

This High Availability (HA) solution requires multi-path capabilities. This is not always available for every piece of software. I want to keep things simple.

A Solution

A solution is to move from a physical network with multiple paths and redundant capabilities to virtual networking.

Each node will have two physical network interfaces. The interfaces will route using OSPF. This is a quick response system that will find other paths if one link or router fails. This provides the HA I want for the network.

Each node will have two VPCs for the ceph cluster, one or more VPC for each container system, and one or more VPC for each VM cluster. A VPC is a “virtual private cloud” It is a virtual network with only allowed traffic.

You can have multiple networks on a single physical network. For example, you can have 192.168.0.0/24 be your “regular” subnet and 172.16.5.0/24 be your data plane subnet. A network interface configured as 192.168.0.7 will only “hear” traffic on subnet 192.168.0.0/24.

But you can configure a network interface to hear every packet. Allowing a node to “spy” on all traffic.

With a VPC, there is only subnet 192.168.0.0/24 on the one VPC and only 172.16.5.0/24 on the other. Packets are not switched from one VPC to the other. You need a router to move data from one VPC to another. And the two VPCs must have different subnets; otherwise the router doesn’t know what to do.

OVN Logical Switch

It turns out that a VPC is the same as an OVN logical switch. Any traffic on one logical switch is restricted to that switch. You need to send traffic to a logical router to get the traffic in or out of the VPC.

Since the traffic is going through a router, that router can apply many filters and rules to protect the VPC from leaking data or accepting unwanted data.

I configured 4 VPCs for testing. DMZ is part of the physical network. Any virtual port on the DMZ VPC is exposed to traffic on the physical network. This is how traffic can enter or exit the virtual clouds.

The second VPC is “internal”. This is a network for every physical node to exist. By using the internal VPC, each node can communicate with each other, regardless of the physical topology.

That was working.

There was a data plane VPC and a management VPC. Those VPCs were connected to the DMZ through a router. The router is distributed across multiple nodes. If one node goes down, the other node is ready to take up the traffic.

Falling way back

I now have a VPC for testing. The idea is to test everything extensively before moving any nodes to the virtual network. I need to be able to reboot any node and have everything still function.

The VPC came up perfectly. My notes made it easy to create the VPC and configure it.

The problem began when I added a router to the VPC.

Now I can’t get traffic to flow to the VPC.

WTF?

Light is a topic that’s come up a few times in my prepper discussions, and so I thought it deserved its own article. Light is defined as “…something that makes vision possible.” (Merriam Webster) We call the light part of our day, aptly enough, daytime. We can see to study, to teach, to learn, and to do. For hundreds and thousands of years, light has been of immense importance to human beings.

The first light was, of course, the sun. Light happened during the day, and night was when you huddled together and tried not to get eaten. The purposeful use of fire is the second light. Fire allowed us to do things in the dark. It kept predators at bay. It warmed our bodies, our food, and even our souls. Once fire was harnessed, it was more a matter of what method to invent to keep the light on at night, safely and effectively.

Some of the earliest lanterns were Canaanite oil lamps. These were basically pinch pots, or rather, a shallow bowl with a pinched spout at one end for holding a wick. These open candles go back farther than our written history. I can say with authority that they do work, too, having both made and used some myself. From a prepping standpoint, open candles can be made out of just about anything. A wick in a bowl works, especially if you can float the wick somehow. Consider an old cork with a hole in it to hold a wick, floating on an oil source.

Fuel oil in early times came from animal fat, or from nut oils. Tallow candles were used as early as 500 BCE, in Rome. They continued to be used until modern paraffin became easily available. Paraffin wax was quickly adopted, because it had almost no smell at all. Candles made from tallow or other animal fats were quite stinky.

Read More

I love this soup, and it can be made with just the squash, or with the pears as well. I think the sweetness of the pears really adds a depth to the soup that the squash alone doesn’t have. If you want to impress your family, serve this in a hollowed out large squash (such as a pumpkin or hubbard squash). What a centerpiece!

Ingredients:

• 2 lbs butternut squash
• 3 tbsp butter
• 1 onion, diced
• 2 cloves garlic, minced
• 2 tsp minced fresh ginger root
• 1 tbsp curry powder
• 1 tsp salt
• 4 cups chicken broth
• 2 Bartlett pears, peeled, cored, chopped into 1″ cubes
• ½ cup half and half

Preheat your oven to 375F. Line a rimmed baking sheet with parchment paper. Cut your squash in half lengthwise, remove the seeds and stringy bits. Place the squash halves, cut sides down on the prepared baking sheet. Roast in your preheated oven until very soft, about 45 minutes. Allow the squash to cool slightly, and then scoop out the meat into a bowl. The skin can be discarded.

Melt the butter in a stout soup pot over medium heat. Stir in the onion, garlic, ginger, curry powder, and salt. Cook and stir until the onion is soft, about 10 minutes. Pour the chicken broth into the pot, and bring it to a boil. Stir in the pear chunks and the reserved squash, and simmer until the pears are very soft, about 30 minutes.

Puree your soup into a smooth liquid. This can be done in a blender, but you must be careful and do it in batches with the pitcher only half full. I prefer to use a stick blender, which can be done right in the pot. You could also use a food processor, but I’d be careful not to have back splash.

Stir in the half and half, and gently whisk to combine. Reheat the soup before serving, if necessary.

If you want to make this pretty, to each bowl add a drizzle of heavy cream over the top and swirl gently with a spoon to make circles of white. Add a sprig of mint or oregano and a couple of pomegranate seeds, and serve.

Notes:

When I make this, I always double or triple the amount of curry I use. You want to be able to really taste the curry flavor. If you have people who are unable to tolerate heat, try using a shwarma blend of curry spices.  It packs a hearty flavor punch without burning the lips off. A spicier curry powder can be added at table, or red pepper flakes, for those who like heat. Alternatively, you can sprinkle a good quality chili pepper oil over the surface instead of cream.

An alternative and more meaty approach to this recipe is to add in leftover Thanksgiving turkey, and serve it the following day with leftover rolls or sliced bread.