I've put together a Linux/KVM system based around an X10SRL-F motherboard, and I'm stumped on what I can only describe as "extremely slow or sluggish video". What I mean by this is that when the system boots, the console messages that appear on the screen scroll excruciatingly slowly, and this adds something like 90-120 seconds to the boot process. My background is in routers and switches that use serial consoles, and the only way I can describe it is like watching a 2400 baud terminal scroll up and down -- it's so slow that you can almost read every line as it appears. Once the machine boots into the graphical desktop, the video is STILL very sluggish, you can visually see that it takes a long time to redraw or resize windows or paint the initial screens of any application that loads.
I don't *think* it's related to the boot/console parameters in any way, because it behaves the same way regardless of whether it's booting (i.e. "console") or already in the graphical desktop.
This is my first IPMI capable board, so I'm pretty sure there is something in between the main system and the IMPI video that's slowing things down, but I don't have the first idea where to look for this. I couldn't come up with a better way to describe in words the difference between the "fast" video and the "slow" video, so I just recorded a couple of short video clips and put them up on google drive: it's very obvious by watching and comparing them how much difference there is in how fast the video is drawn.
X10SRL-F video - Google Drive
Interestingly enough, once I get the system booted, I can do a kexec-reboot (error10/kexec-reboot) and once the system comes back via that approach, it's WAY more responsive. I can't figure out if there's some kind of driver/accelerator that doesn't get loaded the first time around or something like that? I could almost convince myself that if it needed to get booted into the desktop to load the driver this would make sense, but even the first shutdown (i.e. once you've gotten into the Linux desktop) it's still very slow.
To break it down step by step with speeds, it's something like this:
On this machine, one of the last messages to hit syslog as the system boots is the various ports in the bridge going into forwarding state... you can see there's almost a three minute difference between when the video is "good" and things move quickly compared to when it's "bad" and they move really slowly...
Jan 25 12:02:12 lwobker-vms kernel: [ 15.804309] bridge1: port 1(eno1) entered forwarding state
Jan 25 12:02:16 lwobker-vms kernel: [ 19.868160] bridge1: port 2(ens2) entered forwarding state
Jan 25 12:02:18 lwobker-vms kernel: [ 21.640799] bridge1: port 3(vnet0) entered forwarding state
Jan 25 12:03:16 lwobker-vms kernel: [ 16.784198] bridge1: port 1(eno1) entered forwarding state
Jan 25 12:03:19 lwobker-vms kernel: [ 20.284144] bridge1: port 2(ens2) entered forwarding state
Jan 25 12:03:21 lwobker-vms kernel: [ 22.265671] bridge1: port 3(vnet0) entered forwarding state
Jan 25 14:52:47 lwobker-vms kernel: [ 12.746976] bridge1: port 1(eno1) entered forwarding state
Jan 25 14:52:53 lwobker-vms kernel: [ 19.062700] bridge1: port 3(vnet0) entered forwarding state
Jan 25 14:52:54 lwobker-vms kernel: [ 19.900282] bridge1: port 2(ens2) entered forwarding state
Jan 25 14:28:08 lwobker-vms kernel: [ 154.754752] bridge1: port 1(eno1) entered forwarding state
Jan 25 14:28:08 lwobker-vms kernel: [ 160.380175] bridge1: port 2(ens2) entered forwarding state
Jan 25 14:28:39 lwobker-vms kernel: [ 193.922273] bridge1: port 3(vnet0) entered forwarding state
Jan 25 14:36:09 lwobker-vms kernel: [ 153.044693] bridge1: port 1(eno1) entered forwarding state
Jan 25 14:36:09 lwobker-vms kernel: [ 158.652181] bridge1: port 2(ens2) entered forwarding state
Jan 25 14:36:40 lwobker-vms kernel: [ 194.898956] bridge1: port 3(vnet0) entered forwarding state
Jan 25 14:49:21 lwobker-vms kernel: [ 146.123308] bridge1: port 1(eno1) entered forwarding state
Jan 25 14:49:21 lwobker-vms kernel: [ 152.252138] bridge1: port 2(ens2) entered forwarding state
Jan 25 14:49:44 lwobker-vms kernel: [ 187.832669] bridge1: port 3(vnet0) entered forwarding state
Help!?!?!
I don't *think* it's related to the boot/console parameters in any way, because it behaves the same way regardless of whether it's booting (i.e. "console") or already in the graphical desktop.
This is my first IPMI capable board, so I'm pretty sure there is something in between the main system and the IMPI video that's slowing things down, but I don't have the first idea where to look for this. I couldn't come up with a better way to describe in words the difference between the "fast" video and the "slow" video, so I just recorded a couple of short video clips and put them up on google drive: it's very obvious by watching and comparing them how much difference there is in how fast the video is drawn.
X10SRL-F video - Google Drive
Interestingly enough, once I get the system booted, I can do a kexec-reboot (error10/kexec-reboot) and once the system comes back via that approach, it's WAY more responsive. I can't figure out if there's some kind of driver/accelerator that doesn't get loaded the first time around or something like that? I could almost convince myself that if it needed to get booted into the desktop to load the driver this would make sense, but even the first shutdown (i.e. once you've gotten into the Linux desktop) it's still very slow.
To break it down step by step with speeds, it's something like this:
- cold boot the machine. IPMI, BIOS, all that stuff operates at normal speed/responsiveness.
- once the main OS (Ubuntu 18.04.1) starts loading, the boot console messages get very sluggish/laggy and the system crawls along through the boot process. It's important for me to note that the video is synchronous, it's NOT like the system is booting normally and the video is just dropping a bunch of frames along the way... because the video is so slow it bogs down the entire boot process -see logs below... (google drive video link: you can really see the impact at about the 25 second point)
- The main desktop loads, but the video is still super laggy and windows take forever to paint. (video)
- if I do a regular reboot (or shutdown) the behavior stays the same... even the shutdown console is still really slow (video)
- BUT, if I do a kexec-reboot (bounce just the kernel) then it comes back a lot faster. (video after kexec-reboot)
- after the kexec-reboot when the desktop loads, it's quite responsive and snappy! (video)
On this machine, one of the last messages to hit syslog as the system boots is the various ports in the bridge going into forwarding state... you can see there's almost a three minute difference between when the video is "good" and things move quickly compared to when it's "bad" and they move really slowly...
Jan 25 12:02:12 lwobker-vms kernel: [ 15.804309] bridge1: port 1(eno1) entered forwarding state
Jan 25 12:02:16 lwobker-vms kernel: [ 19.868160] bridge1: port 2(ens2) entered forwarding state
Jan 25 12:02:18 lwobker-vms kernel: [ 21.640799] bridge1: port 3(vnet0) entered forwarding state
Jan 25 12:03:16 lwobker-vms kernel: [ 16.784198] bridge1: port 1(eno1) entered forwarding state
Jan 25 12:03:19 lwobker-vms kernel: [ 20.284144] bridge1: port 2(ens2) entered forwarding state
Jan 25 12:03:21 lwobker-vms kernel: [ 22.265671] bridge1: port 3(vnet0) entered forwarding state
Jan 25 14:52:47 lwobker-vms kernel: [ 12.746976] bridge1: port 1(eno1) entered forwarding state
Jan 25 14:52:53 lwobker-vms kernel: [ 19.062700] bridge1: port 3(vnet0) entered forwarding state
Jan 25 14:52:54 lwobker-vms kernel: [ 19.900282] bridge1: port 2(ens2) entered forwarding state
Jan 25 14:28:08 lwobker-vms kernel: [ 154.754752] bridge1: port 1(eno1) entered forwarding state
Jan 25 14:28:08 lwobker-vms kernel: [ 160.380175] bridge1: port 2(ens2) entered forwarding state
Jan 25 14:28:39 lwobker-vms kernel: [ 193.922273] bridge1: port 3(vnet0) entered forwarding state
Jan 25 14:36:09 lwobker-vms kernel: [ 153.044693] bridge1: port 1(eno1) entered forwarding state
Jan 25 14:36:09 lwobker-vms kernel: [ 158.652181] bridge1: port 2(ens2) entered forwarding state
Jan 25 14:36:40 lwobker-vms kernel: [ 194.898956] bridge1: port 3(vnet0) entered forwarding state
Jan 25 14:49:21 lwobker-vms kernel: [ 146.123308] bridge1: port 1(eno1) entered forwarding state
Jan 25 14:49:21 lwobker-vms kernel: [ 152.252138] bridge1: port 2(ens2) entered forwarding state
Jan 25 14:49:44 lwobker-vms kernel: [ 187.832669] bridge1: port 3(vnet0) entered forwarding state
Help!?!?!