Server Down for Migration (Should be Back in an hour)
-
We're migrating our main back-end server to a new host that should hopefully prevent issues like the unscheduled outage that happened a week ago.
This all stems from the Intel "Meltdown/Specter" vulnerability you might have heard about in the news. Our server host, Linode, patched all their machines, but the patch seems to have caused instability at the CPU level and occasionally the servers would lose all internet connectivity (the intel networking would crash). Linode has been working on fixes for this and they are migrating us to a new host which has some fixes applied to prevent that from happening.
-
I think there were reports of a pretty big CPU hit, so perhaps the CPUs in the servers are some of the more impacted? (I assume the hosting in on virtual servers making the whole thing more complicated too, I was just a hardware or SMB MS server gimp so not really my area)
-
@smashman42 J-Novel Club isn't particularly CPU intensive, and since it runs on node (which is single threaded) I doubt it would impact performance for us. The problem is that intel's patches to fix the security bug seems to have introduced rare issues that lock the network stack on the processor? Linode said that they have evidence this is the case but it's rare enough (it happened to us once in the span of 2 weeks) that they've had trouble pinpointing the exact cause/solution.
The new server we are migrating do has some experimental fixes applied, so hopefully it will be nice and stable. We might need to migrate again however in the future.
-
And we're back!
