Skills

I’m exhausted. I’ve been pulling fiber for the last two days. All part of an infrastructure upgrade.

Normally, pulling cable in a modern datacenter is pretty easy. This is not a modern datacenter.

The original cable runs were CAT6 with RJ45 connectors. When the cables were installed, the installation had to be nondestructive. No holes in walls, no holes in floors. Hide the cables as best you can.

One of the cables we removed was to a defunct workstation. It had been run across the floor and then covered with a protective layer to keep it from getting cut or snagged. The outer insulation had been ripped away. There was bare copper showing. Fortunately, that particular workstation hasn’t been in place for a few years.

The backbone switch was mounted in the basement. Not a real issue. The people who pulled some of the last cable didn’t bother to put in any cable hangers. So it had loops just dangling.

There were drops that could not be identified. Those are now disconnected, but nobody complained, so nothing was taken offline.

I’ve found a new favorite cable organizer.

Cable Management Wire Organizer

These are reusable. They open fully and will hold many cat6 and even more fiber. They have the 3M foam double-sided tape on them. This works great against smooth, clean surfaces.

The place where they shine is that they also have a hole designed for a #6 screw. In places where there were no smooth surfaces, much less clean surfaces. The sticky held them in place long enough to drive a screw.

There are no more dangling cables.

My only hope is that there are no more configuration issues with the new switch. *caugh*DHCP*caugh*

It seems appropriate to talk about what to do after a flood, right now. With all the devastation down in the southern states, it’s on all our minds.

First and foremost, if you’re in the path of something and you know about it, listen to locals. If you’re asked to evacuate, DO SO. The people in the Carolinas and Tennessee didn’t receive evacuation orders until after it was too late, but people in Florida knew what was coming. Those who chose to stay behind did so at their own peril. When you ignore an evacuation order, you are telling the authorities that you are entirely on your own, and do not expect any aid from them. You do so knowing that 911 won’t work, that health and medical emergencies will need to be dealt with in house, and that you’ve decided you know better than the weatherman.

I realize it can be hard to leave everything behind. I’ve done it twice in my life, and it wasn’t any easier the second time than it was the first. What you’re leaving behind is Stuff and Things, and I guarantee you, Stuff and Things are not nearly important as Friends and Family. I speak with experience and authority here. Leaving ahead of an emergency allows you to leave on your own terms.  You can take the time to pack bags, to grab family photographs and important documentation. You can bring along family pets, and favorite children’s toys. Leaving (or attempting to leave) once the emergency is upon you means you’re leaving on the emergency’s terms, rather than your own. You may not have time to find the cat cowering in the basement, or find the woobie your kid can’t sleep without.

If you can’t leave, for whatever reason, then you need to make reasonable plans for what is likely to happen, what could happen, and what is unlikely to happen. There will always be things that can’t be planned for (like the current destruction of Asheville, for example), but you have to do your best.

In my area, we deal with freezing rain often enough that we know the power will go out at least once, and likely twice over the winter. In the spring, it’s wet weather and high winds knocking down trees and taking out power lines. The most likely occurrence is that we’ll be without power for a day or less. What might happen is that we’re out of power for a couple of days, and our water might be affected. It’s unlikely that we’ll be out for more than a few days, but we have things on hand to make that less troublesome if it does happen. We’ll be very busy canning all the stuff in our freezer, honestly.

Read More

There are foods that I consider to be autumn foods. Squash, pumpkin, sage sausage, and soups in general come to mind. I love cooking seasonal foods, partly because it tends to be a cheaper way of eating, and partly because there’s just something scrumptious about picking up a squash that was in a field 2 hours ago, and baking it up. Today’s recipe is for a stuffed squash, but I haven’t told you what type. The honest answer is, it doesn’t really matter. I found Sweet Dumpling Squash on sale today, but had originally gone looking for acorn. I will say that Spaghetti and Butternut Squash don’t do as well in this recipe, only because they don’t have a big hole in them to stuff full of yumminess. But if that’s what floats your boat, go for it! You do you, Boo!

Ingredients:

• 2 winter squash, halved and cleaned of seeds
• 1 lb ground sausage, sage/breakfast style
• 1 small onion, diced
• 1 small carrot, diced
• 1 rib celery, diced
• 2 to 5 cloves garlic, minced
• 1/2 cup of COOKED rice, barley, quinoa, or other grain
• spices, to taste (either sweet or savory, your choice)
• cheese, for topping (optional)
• walnuts, for topping (optional)

Preheat your oven to 400F. Place the four halves of squash, open side down, onto a rimmed baking sheet. Add a bit of water (this helps by steaming the flesh of the squash slightly), and put in the oven for 30 minutes to an hour. Check after 30 minutes for doneness; you want the squash to be soft through, but not brown or caramelized at all.

While the squash is cooking, brown up your ground sausage in a cast iron pan or pot. I prefer to use a nice sage sausage for this, because it’s my go-to flavor for this time of year. If you prefer sweet or hot Italian, go for it. If you don’t do pork, feel free to substitute in ground beef, tofu, TVP, venison, squirrel, turkey, or chicken. If you’re not using a sausage, pick up a good “sausage spice” to add to your ground meat, or it will be bland later. You don’t want bland.

When the sausage is thoroughly browned but not yet crispy, use a slotted spoon to remove it from the pot. Leave behind as much of the fat as you can (add olive oil if there isn’t enough in the pan). Toss in your onions, carrots, and celery, and saute until they’re just soft and starting to brown up. Add in the garlic, and saute until it releases its oils, about 30 seconds to a minute. If there’s a decent amount of meat or veg stuck to the bottom of the pan, deglaze with a splash of red wine, using a wooden spoon to mix thoroughly. Add in the cooked grain, and stir to incorporate. The grain should soak up any excess liquid. Add in the meat, and then spice to your taste.

When it comes to spicing this, you can go either savory or sweet. For savory, consider salt and pepper, oregano, thyme, and sage. If you go sweet, start with salt and pepper, then add in a bit of sweet paprika (get a GOOD type like Szeged rather than using glorified red dyed sawdust like McCormick). Then stir in either a bit of maple syrup (my favorite) or some dark brown sugar. If you go the sweet route, also consider tossing in some raisins and dried cranberries or cherries for a delicious flavor burst.

Once the squash is ready, fill the hole with your meat mixture, adding an equal amount to each half. Top with cheese if you’ve decided on savory, or nuts and berries if you’re going sweet. Return to the oven and bake at 400F for 15 minutes or so, just until everything is hot through and the cheese, if used, is thoroughly melted.

Notes:

If you have picky kids who are likely to eat the filling and then ignore the squash, do what we did: cut the squash up and add the filling to it. Make it into a casserole! This way, they have to eat the squash to get at the “good bits” and you get to feed them well. 😉

This is one of those “do whatever you like” recipes that changes every time I make it. I’ve made vegan versions with Beyond Sausage, or with diced up vegan Field Roast sausages, or with tofu. I’ve made non-pork versions for friends who don’t do pig, and it’s turned out beautiful. If you want it to be a side dish, simply skip the protein entirely and stuff the squash with your grain and whatever veggies you have on hand. Shredded brussel sprouts are pretty good in this, especially when mixed with a Tex-Mex wild rice blend and raisins.

Enjoy!

Part of the task of making a High Availability system is to make sure there is no single point of failure.

To this end, everything is supposed to be redundant.

So let’s take the office infrastructure as a starting point. We need to have multiple compute nodes and multiple data storage systems.

Every compute node needs access to the same data storage as all the other compute nodes.

We start with a small Ceph storage cluster. There are currently a total of 5 nodes in three different rooms on three different switches. Unfortunately, they are not split out evenly. We should have 9 nodes, 3 in each room.

Each of the nodes currently breaks out as 15 TB, 8 TB, 24 TB, 11 TB, and 11 TB. There are two more nodes ready to go into production, each with 11 TB of storage.

It is currently possible to power off any of the storage nodes without effecting the storage cluster. Having more nodes would make the system more redundant.

Unfortunately, today, an entire room went down. What was the failure mode?

DHCP didn’t work. All the nodes in room-3 were moved to a new 10Gbit switch. Actual 4×2.5 2×10. The four 2.5Gbit were used to connect three nodes and one access point. One of the 10Gbit SFP+ ports was used as an uplink to the main switch.

When the DHCP leases expired, all four machines lost their IP addresses. This did not cause me to loss a network connection to them because they had static addresses on a VLAN.

What did happen is they lost the ability to talk to the LDAP server on the primary network. Because they had lost that primary network connection, no LDAP, no ability to log in.

The first order of repair was to reboot the primary router. This router serves as our DHCP server. This did not fix the issue.

Next I power cycled the three nodes. This did not fix the issue.

Next I replaced the switch with the old 1Gbit switch (4x1Gbit, 4x1Gbit with PoE). This brought everything back to life.

My current best guess is that the cat6 cable from room 3 to the main switch is questionable. The strain relief is absent and it feels floppy.

More equipment shows up soon. I’ll be pulling my first fiber in 25 years. The new switch will replace the current main switch. This is temporary.

There will be three small switches for each room. Then there will be a larger switch to replace the current main switch. The main switch will be linked with 10Gbit fiber to the 3 rooms in server rooms. The other long cables will continue to use copper.

Still, a lesson in testing.

The final configuration will be a 10Gbit backbone with OM4 fiber, the nodes will be upgraded to have 10Gbit NICs which will attach to the room switches via DAC cables. There will then be a 2.5Gbit copper network. The copper network will the default network used by devices.

The 10Gbit network will be for Ceph and Swarm traffic.

I’m looking foward to having this all done.

I’m all about prepping canned and dehydrated food in advance of any emergency. I’m more interested in learning to garden than in having 20 years of supplies on hand. My goal is to have enough food (“food” being defined as the proper ratio of fats, carbohydrates, and proteins to keep a person not just alive, but thriving) in my pantry that I can last for 18 months. That’s the longest time I would be without a garden harvest. These are good plans, but I also like to know what I can get from the world around me.

I live in New Hampshire, and there are lots of things to eat that can be foraged easily. Starting with the easiest, there are blackberries, raspberries, and blueberries pretty much all over the state. They’re largely free for the taking, and there aren’t any “look alike” poisonous versions. Eastern Hemlock is a type of fir tree that is indigenous to the area. You can tell it by its flat spines/leaves (see the picture). It contains a lot of Vit C, and makes a very nice tea. In the spring, dandelion leaves are a great source of fresh greens. While a little bitter, they’re full of vitamins and minerals, and can be added to salads and soups. Don’t forget fiddleheads, though they’re only available in the spring for a very short amount of time. Cattail (you know, the “corndog” like things growing in ponds and slow moving rivers) is also edible. Different parts of the plant are good at different times of the year.

There are many mushrooms that are edible, but all mushrooms come with a caveat. Mushrooms, all of them, are edible… but some are only edible once. Meaning, there are poisonous mushrooms, and ones which will mess with your gut in bad ways even if they don’t kill you. As an example, I found a stand of amazing looking fungi at the base of a tree in my back 40. I asked a friend of mine, who is an expert mushroom hunter, if they were edible. They looked very much like an edible mushroom I know about. She laughed when she saw the image, because the shroom in the picture is more of a “draino” style medicine. Basically, if your stomach needs to be cleaned out entirely, it’s a really good mushroom to eat. It doesn’t harm you, but it goes right through your system, taking everything with it. I’m glad I didn’t make those into mushroom soup…

Read More

I made this one this past week, and was thoroughly impressed by it. The flavors were good, though it needed a bit of tweaking to make it perfect. It was a VERY good “dump meal,” meaning all I had to do was stuff things in the crock pot and set it to high. I’ve been sick as a dog this week, and needed stuff that was simple to make. This fit the bill, and was healthy and tasty as well!

Ingredients

• 1-1/2 cups barley
• 1 lb mushrooms, sliced
• 2 lbs boneless, skinless chicken (thighs, breasts, or a combo)
• 4 cups chicken broth
• 1/4 cup lemon juice
• 1/4 cup dry white wine
• 6 garlic cloves, smashed and chopped
• 1 tsp dried thyme
• 2 tsp salt
• 1 tsp onion powder
• ½ tsp red pepper flakes
• black pepper, to taste
• 10 oz frozen sweet peas
• 1/4 cup creme fraîche, sour cream, or plain yogurt (dairy free works well)
• 2 tbsp Dijon mustard, plus more to taste
• 1/2 cup fresh tarragon or dill, minced
• Parmesan cheese

In your crock pot, combine all the ingredients from barley to the black pepper. Stir well to combine, and then cook on high for about 3 hours. Stir again, making sure to immerse the chicken in the liquid. It’s going to seem a bit soupy, but trust in the recipe. Cook for another hour on high, or switch to low to cook for up to another 3 hours.

Remove the chicken to a plate and let sit for a few minutes while you do other things. Stir in the frozen peas and let them heat for about 2 minutes.

While the peas are warming, whisk together the creme fraîche and mustard. Taste, and add more mustard to suit your palate. Fold this mixture into the barley and mushrooms in the crock pot. Now, shred or pull apart your chicken. It should fall apart easily. Return the chicken to the pot, then taste to see if you need any more salt or pepper. Stir in the chopped fresh herbs, and serve immediately.

Add the Parmesan at the table, for those who wish it.

Notes:

I found the tarragon to be a bit strong in flavor, even though I only used about a tablespoon of it. The slight liquorish flavor just overpowered the rest of the dish. I would omit this next time, and use either all dill, or a mix of dill and Italian Seasoning instead. Another suggestion would be Herbs de Provence.

Even though you’re only adding about a half cup total of “creamy” stuff to this dish, it comes out incredibly creamy tasting. I used a cashew based yogurt, because I’m not able to eat dairy, and it was delicious. I used a smooth traditional Dijon, and I think I might try a more stone ground style next time. This was great as a stand-alone, and could easily be put together in a morning before running out the door. I think it would also be nice paired with a light salad.

Following the basic rules of three, we can live without water for three days. That’s not entirely true, but it’s close enough. Water is largely what we’re made of, being between 50% and 60% of our makeup. Water keeps our brains working, flushes toxins out of our bodies and protects our muscles, helps lubricate our joints, and keeps the blood flowing through our veins. We can become dehydrated incredibly quickly, depending on our level of activity, the ambient temperature, and the humidity, so it’s important to keep drinking. A general rule of thumb is, if you’re feeling thirsty, you’re already well on the way to being badly dehydrated.

It is possible to become dangerously dehydrated within 24 hours. Dehydration is when you take in less water than your body is using up in sweat, tears, urination, etc. and it’s very serious. While you can be on day 2 or 3 of not having water and still survive it, it will be a painful survival. After that first day, you are not going to be thinking well, because your brain will not be functioning as easily. You will be sluggish. Your mouth will be dry, and possibly your nose, and skin. You become exponentially more likely to have seizures or experience hypovolemic shock (when your blood volume drops below a certain level). This all has to be kept in mind when you’re dealing with a SHTF scenario. Drinking is going to be incredibly important.

So how do you keep water on hand? That’s something you have to decide for yourself. I live in a water rich area of the world, and so I don’t store a lot of water. I keep enough on hand for 72 hours of drinking and washing (of myself and my dishes), which is about 2 to 3 gallons per person per day. Instead of keeping lots of water on hand, I keep water purification items on hand. I have the means and knowledge to get water, but I’ll need to purify it. To purify water, you have to either boil it, disinfect it, filter it, or treat it with UV light. For safety’s sake, I recommend doing at least two of the four methods, though depending on your emergency, that may change.

Read More

Chris wanted me to share my sauce from last night, so I said heck with it. I liked it enough that I’m going to post it up here now.

To start off, this was my first attempt at making a Bolognese from scratch. I’ve always used jarred sauces and then “spiffed them up” in the past. I was in the mood to try something more challenging, but not too much because I’ve been so sick this week. The consistency was good, the amount of meat to sauce was good, the mouth feel was excellent, and the taste was incredible!

Ingredients:

• 1 lb ground pork (not flavored, just plain)
• 1 celery stick, finely diced
• 1 medium carrot, finely diced
• 1 onion small, finely diced
• 2-3 garlic cloves, minced
• 1 tbsp olive oil
• 2 14.5 oz cans of crushed tomatoes or stewed tomatoes 
• 1 bay leaf
• 1 tbsp tomato paste
• 1 tbsp Worcestershire sauce
• 1 tsp dried oregano
• 1/2 cup whole milk (or oat milk for dairy free)
• salt and pepper to taste
• grated Parmesan cheese for serving

Heat your olive oil in the bottom of a Dutch oven or saucepan, and add the ground pork. Stir regularly until it’s almost cooked through, about ten minutes. While the pork is browning, mince up your vegetables. Add these to the meat, once it’s no longer pink. Continue to cook on medium heat until the onions begin to soften and clarify.

Add in your tomato paste, Worcestershire sauce, and garlic. Cook for another 3 to 4 minutes, continually stirring. Reduce the heat to the lowest setting and add in the tomatoes, bay leaf, and dried oregano. Stir it together, and partially cover the pot. Simmer this on your lowest heat setting for 45 to 60 minutes. Stir the sauce every 15 minutes, to make sure it doesn’t stick to the bottom. Technically, this can continue to simmer for as long as you like. You want it to be fairly thick, so after an hour you may want to take the lid off, and stay there, stirring, while it thickens. If you’re in a rush, you can remove any excess liquid with a spoon or small ladle.

Turn off the heat, then add in the milk. Stir it together, taste, and then add salt and pepper to taste. If the sauce has a metallic or acid tang, add a half teaspoon of brown sugar to it. While you can skip adding the milk entirely, it really does add a depth to the sauce that makes it velvety. It also helps mellow out the tomato acidity.

Notes:

The original recipe calls for crushed tomatoes, but I didn’t have those. I could have stuck my stewed tomatoes into the blender and whizzed them up, but I chose to use the tomatoes chunky. I’m glad I did. Many of them broke up during the cooking of the sauce, but the pieces of tomato really added a lovely mouth-feel, and a certain “umami” to the final sauce.

This goes perfect over any kind of pasta, but it’s best over one that has a rough texture and lots of gripping surface. Think bowties or rotini, or even radiatori.

Serve this one up with a slice of hearty french bread, or some garlic knots. Don’t bother with salad; it’s much too meaty for that.

There are three ways of making pasta shapes. You can extrude the dough through shaped holes, cutting the forms off as needed. You can roll the dough thin, then hand cut the dough to be shaped. The same can be done using a “pasta machine” which rules the dough to a known thickness. Or you can use cutters to cut sheets of pasta.

I’ve been making pasta dough for a couple of years. I stepped up my game recently by deciding to roll my dough instead of squeezing it in the pasta machine.

Since I was going to roll out the dough, I decided to get rid of my mixing bowl and do it on my board with the well method.

This is a step forward. One of the lessons I’m still learning is how small a batch can be made this way. If the batch is not large enough, there isn’t enough floor on the board to create a well with deep enough walls.

I was lucky, there were only two eggs in play, so the eggs didn’t go everywhere, just a slight panic.

Which brings us back to dough. Normal pasta dough is an egg and floor mixture. My recipe is 3/4 of flour to 1 large egg. Occasionally, it is two eggs plus a yolk. It really depends on how the dough feels.

Today, the dough was too wet. I should have added more flour, but I didn’t.

The eggs are mixed into the flour until it comes together. At that point, I switch to kneading. I knead for 10 minutes, by the clock. The sad thing is that I always check the time remaining at 2:30. I don’t know how I get there, but I do.

This egg pasta works well for rolling out or squeezing. This was turned into rigatoni.

I didn’t know I could make this by hand, I thought I would need to use an extruder for rigatoni. I am happy with my results.

It does not look like machine made pasta. The dough was a little wet. I also had some issues cutting it. Next time I will use the bicycle cutter to get nice squares.

Ally took the other half of the egg pasta and made beautiful bow tie pasta.

The other type of pasta dough is made with water and semolina flour. Bluntly? I just guessed at the amount of warm water to add. It came out nearly perfect.

This pasta dough doesn’t roll out as flat as the egg pasta, but it is used for other pasta shapes. We started with Orecchiette. Those were easy to make, but I didn’t like them. Ally loved making them, so I let her do it.

This is made by cutting a 1/2 diameter rope of dough into 1/2 long chunks. Then you roll the under your thumb.

Malloreddus is made the same way, but the pasta is pressed on a grooved board to make a wonderful pattern. This was my favorite shape to make. They go fast, they look nice, and they have a good bite to them. Plus they hold the sauce.

The Busiate was another fun shape. It uses a 1/4 in rope of dough cut into 3 inch long sections. These are twisted around a skewer and the rolled flat.

I think they look delightful, I want to practice them some more.

I’m hoping that Ally will do an article about the sauce she made for our homemade pasta. I will say it is the best I’ve had in years.

Stew is the bedrock of my winter meal plan. Part of my family dislikes soup, but is okay with something thicker. Stew is the way to go! Stew is basically a protein (chicken, beef, lamb, etc), a fat (bacon grease, olive oil, veg oil, etc), vegetables (onions, carrots, celery, etc), and a starch (rice, beans, potatoes, etc). In whatever combination you like, mix together, and you have stew. Of course there’s a bit more to it, but that’s the basic “stew recipe.”

Because stew was usually made with leftovers or scraps, the little bits that weren’t enough for another full meal, we don’t really have written stew recipes.  A good stew is different every time you  make it, because the ingredients will never be exactly the same. That is the right way to do it! So let’s make a nice, hearty chicken stew.

Ingredients

• a cut up chicken, or several thighs, bone in and skin on
• 1 or 2 onions or leeks, rough chopped
• 3 large carrots, coined
• 2 ribs of celery, diced
• 4 cloves garlic, rough chopped
• white wine (optional)
• 1 purple top turnip, diced
• 2 parsnips, coined
• 2 to 3 cups shredded cabbage
• 1 cup barley, well rinsed
• spices (salt, pepper, oregano, parsley, dill, sage, rosemary, etc.)
• oil for searing
• water to fill the pot

In a large soup or stock pot, add a drizzle of oil (olive, safflower, or vegetable) and bring up to a medium heat. Add the chicken and brown it all over, then remove it from the pot (set it aside on a plate). Add the carrots, onions, and celery, and cook until the onions soften and begin to brown very slightly. While they’re cooking, you may need to add more oil. This mixture of carrots, onions, and celery is called a mire poix, and it’s the basis of most good quality soups and stews. You want to scrape any of the browned chicken off the bottom of the pan (that’s the fond and it’s part of what makes for a rich flavor later). Add in the garlic and saute until they are fragrant, usually less than a minute.

Drizzle in some of the wine (or chicken broth if you don’t do wine), and deglaze the pan entirely. Once that’s done, add the rest of the vegetables and allow them to cook for 2 to 3 minutes. Add all the remaining ingredients, and enough water to cover everything, plus 3 cups (this is to allow for the barley to absorb some of the water – you may want to try a bit more or less, to make it the thickness you prefer). Add in some salt, pepper, and other herbs to taste, and bring to a boil, then allow to simmer for a minimum of an hour.

Check for flavor, then add more spices as necessary. The above are only suggestions, and you’re welcome to try whichever spices you like! Simmer for another hour or more, until the chicken is falling off the bone and shreds easily. You may wish to pick out the skin and bones at this point, or serve it as is. Serve this up with fresh baked bread for a hearty and healthy meal.

Notes:

You can make this recipe with diced chicken breast if you prefer. I like the added flavor and the moistness of the meat when I use bone-in, but not everyone likes bones. I almost always make this recipe with either a full cut up chicken, or with chicken legs with thighs attached. It comes out meaty and delicious!

Recipe by M. Allyson Szabo, author of The Re-Enactor’s Cookbook (available on Amazon).