Serial console broken after 11.2-U1 -> U4.1 update
Description
Problem/Justification
Impact
SmartDraw Connector
Katalon Manual Tests (BETA)
Activity
Kris Moore November 26, 2019 at 1:59 AM
Fix coming in via upstream.
Kris Moore November 26, 2019 at 12:50 AM
can this be closed out?
Edward Pilatowicz June 27, 2019 at 11:52 PM
Hey Former user. My BIOS has a "Redirection After BIOS POST" option that can be set to "Bootloader" or "Always Enable". It was set to "Always Enable". (And has been that way since I built the system in 2017, running different versions of FreeNAS the whole time.) Changing it to "Bootloader" doesn't help, I still get white-space on the console. (And I confirmed this was with the loader.conf.local updates from the legacy UI.) That said, enabling the "Bootloader" setting did break my workout around of manually selecting the "Boot FreeNAS (Serial Console)" bootloader menu option. With these two options combined, instead of getting console output I also got a stream of white-space.
Waqar Ahmed June 27, 2019 at 10:21 PM
Thank you for getting back. Okay so stream of spaces on serial console may mean that your bios still does serial console redirection, fighting with os for serial port. Can you please go to bios settings and check that redirection is set to "loader only". Want to see if this helps. Also please do this with serial console enabled from legacy UI with correct settings.
Cheers!
Edward Pilatowicz June 26, 2019 at 5:46 PMEdited
Hey Former user,
So here's the initial contents of loader.conf.local:
Then I tried going into the legacy UI and configuring the serial console there:
This resulted in an updated loader.conf.local:
Then when I rebooted I got the exact same failure that I saw after the initial U4.1 upgrade. Basically, I see the BIOS messages and bootloader menu, but once the kernel loads all I get is streams of white space on the console.
This seems really weird because if I reboot and choose the "Boot FreeNAS (Serial Console)" boot option (that I modified to use comconsole_port=0x2f8) the console works again, but that menu option uses all the same parameters as what was added to loader.conf.local. Hopefully this makes more sense to you than it does to me.
I've been running 11.2-U1 (and prior versions) for a while now with a serial console configured. The console port is uart1 (0x2f8), which is actually the ipmi SOL console on my SuperMicro A1SRi-2758F-0 motherboard. So I normally access the console remotely via ipmitool. After the update to 11.2-U4.1, I see BIOS and bootloader messages on the console (via BIOS console serial port redirection), but after selecting a boot option and seeing the kernel load, i get a stream of white-space on the screen. The system does eventually boot and I can ssh in, but I can't see any console messages or login on the console.
Looking at the kernel console configuration, everything seems ok:
---9<---
grep -E '^(sio|uart)[0-9]' < /var/run/dmesg.boot
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
uart1: console (115200,n,8,1)
---9<---
Ie, the kernel sees the port and has redirected output to it as expected (and with all the correct line options that worked in the past).
Logging into the GUI and going to System > Advanced, The "Enable Serial Console" option, the "Serial Port" option is set to "---", and if i click on it I don't actually get any ports to select. (The only option displayed is "---".) I tried disabling serial support in the GUI and re-enabling it, but now I can't re-enable the option because it says I must select a "Serial Port", but there are no ports listed to select (only "---" is available). This GUI problem seems like NAS-101984: https://jira.ixsystems.com/browse/NAS-101984 , which is apparently fixed in 11.3.
After disabling the serial port via the GUI, if i try selecting the "Boot FreeNAS (Serial Console)" option via the bootloader, then it seems like the system defaults to uart0:
---8<---
grep -E '^(sio|uart)[0-9]' < /var/run/dmesg.boot
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: console (115200,n,8,1)
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
---8<---
Eventually I worked around the problem by editing /boot/menu-commands.4th and adding:
s" set comconsole_port=0x2f8" evaluate
into the ": serialboot" section of the menu. Then when I reboot if I select "Boot FreeNAS (Serial Console)", my serial console is works again.
I'm wondering what happened to my serial console configuration during the upgrade? If there's any addition debugging information you'd like me to supply just let me know.
Also, I figured out a workaround, but editing /boot/menu-commands.4th feels like a horrible hack. I'm wondering if there's a better way to fix this? Perhaps I could somehow manually update the FreeNAS configuration store to save the proper serial configuration there (bypassing GUI bug )? Alternatively, what would be a good way to make this configuration the default (so I don't have to interactively select a non-default bootloader option every reboot)?