You Get What You Pay For

My first fiber switch turned out to be a L3 managed “switch”. Way cool. But I purchased a cheap switch and found that it completely undocumented.

It has taken me a while to figure things out.

The configuration GUI is an What You See Is All You Get type. There is enough there that you can get the switch up and running, but not enough to fully configure the L3 Switch.

To accomplish that, you need to use the CLI. Not a problem, I like CLI’s.

Of course, there is no documentation but for tab completion and very limited help screens.

I get it mostly working.

After playing with the Free Range Routing Suite (FRR) for a while and getting OSPF working on all of my hosts and the primary router, I was feeling pretty confident.

It seems that FRR took their configuration model almost directly from Cisco’s CLI. The number of times I used a Cisco help page to determine how to configure an OSPF setting is remarkable.

The new L3 switch turns out to have a Cisco like configuration language. And what isn’t Cisco like, is FRR like. Neither Cisco nor FRR, but close.

Today I had a tremendous success, I moved a ceph host from the physical network to the OVN network.

This included moving that segment of the network to a new subnet. And everything sort of worked.

The issue turned out to be a routing issue.

The correct answer is to turn on OSPF within the new physical router. It does support it, after all.

Having played with the damn thing for a few hours, breaking my network multiple times, I was about to give up when I happened to notice a strange value for a setting.

That setting? MTU, of course.

Even though every interface shows an MTU of 9000. Even though jumbo frames are turned on and using a 9000 byte frame.

Even though an MTU of 9000 is very much supported, the MTU of the “VLAN” was set to 1500.

Now, Cisco VLANs are not the same as a tagged VLAN. A tagged VLAN acts like a separate physical network. They are where you place interface settings. These VLANs can then be assigned to a physical port.

The physical port’s MTU overrides the VLAN MTU. This means my jumbo packets from host to host work.

The problem is that the VLAN MTU is maxed out at 2000 bytes. This seems to only affect the OSPF traffic and not the physical interface. But I’m dead in the water or I need to figure out how to do this differently.

Still, I didn’t pay an arm plus a leg for this physical router. I’ll get it to work.