Fujitsu TX1320 M3 - Cheap low power server (barebone)

[Pirrao]

The bios option doesn't work for me, i tried to flash the firmware to 6.20 with the usb dos feature (EFI one doesn't boot, gets stuck on dracut-initqueue). Next will try to move the jumper on the board that says password reset.

if you guys know any solutions i'll be very grateful.

EDIT: Dont use Opera, Internet explorer or chrome, the iRMC will not work, Firefox is the way.

 

[camping8929]

In case anybody missed it, there's a "new" (4 months old at this point) V5.0.0.11 R1.34.0 BIOS version on the Fujitsu website.
The one before it - R1.33.0 - was released on 06/07/2022. The newest one was released on 24/03/2023.

Read the chanelog entry at the bottom of this post - particularly if you use Secure Boot - before updating.

FUJITSU Support pages EMEA

Fujitsu co-creates with customers to help them digitalize with confidence. We provide innovative IT services and digital technologies including cloud, mobile, AI and security solutions. Together with our stakeholders we contribute to the future of society.

support.ts.fujitsu.com

I updated it by copying the contents of the .ZIP archive to the USB stick, physically inserting it into the server and booting into the update through the F12 boot menu. I used USB because I had a feeling this .ZIP is only ever meant for USB installation. I guess you can fiddle with it and try through iRMC but I wasn't able to find any .bin file.

Changelog for R1.34.0 from Fujitsu's product page for TX1320 M3:

PRIMERGY TX1320 M3 / TX1330 M3 uEFI FW changes:

uEFI FW - V5.0.0.11 R1.34.0 for D3373-B1x
==============================================
- Vulnerability: Support for IPU2022.2 (Intel Platform Update).
  Following CVEs are addressed.INTEL-SA-00669: CVE-2022-26074
- Vulnerability: Support for IPU2022.3 (Intel Platform Update).
  Following CVEs are addressed.INTEL-SA-00610: CVE-2022-29466
- Addressed RingHopper / VU#434994/ CVE-2021-33164
- SPS(Server Platform Service) FW update to 04.01.04.700.0
- Updated Certificate for Secure Boot Key. If you are using PK/KEK/DB/DBX
  variables inside of BIOS for Secure Boot, please update to this BIOS version
  and remove registered keys from "Secure Boot Configuration" of BIOS setup
  menu, then register BIOS built-in default keys.
- Solved the issue that option names in help message don't match actual option
  names of "USB Port Security".
- Solved the issue that after the system does not boot from "NextBootDevice"
  to be set via Fujitsu iRMC REST Client Tool.
- Fixed the condition of "Password on Boot" can be selected.
  before:Setting "Adimin Password",after:Setting "User Password"
- Fixed the issue that Fw/Oprom version of Intel Lan cards may incorrect value
  or "N/A" in Network information of iRMC Web-UI. Affected cards are as below.
  X540-T2, I210-T1, I350-T2, I350-T4, X550-T2, X710-DA2
- Solved the issue that "Could not resolve symbol [\_GPE.L69.D1F0],
  AE_NOT Found" is logged with SLES15 SP3.
- Solved the issue that password cannot be authenticated.

 

[Bjorn Smith]

Hi,

I have a weird issue with my TX1320's.

3 of them sometimes power off at the exact same time - which is really not ideal with a 5 node ceph cluster.

They are connected via an extension cord that is hooked into an UPS.
i.e. all 5 server share the same connection to the UPS - so I think that rules out an issue with the UPS - otherwise I would think all 5 would shut down.


(sorry for the poor drawing)


iRMC has this in its logs for all 3 servers:


I am not sure what to make of this - is it the extension that is causing isssue?
3 server's power supplies that crap out at exactly the same time?

It happened first on the 5th of august and the obviously 3 days after - one day after I had left the house for a week's vacation

What would you guys do if you experienced this?

Plug servers direcly into the UPS?
New extension?

Thanks for any anwers/ideas

EDIT: I have moved one of the "failing" servers to a direct connection to the UPS - so if/when it happens again, I should know if its the extension that is causing the issue.

 

[Rand__]

No logs in the ups?
Extension is stupid i suppose?
No shutdown time configured in Bios?

 

[gb00s]

...
It happened first on the 5th of august and the obviously 3 days after - one day after I had left the house for a week's vacation
...

Seems the other two are more psychologically stable when you are not at home. The three just went into depression mode.

Questions would be, ...

1. What UPS & extension are you using?
2. Where are the shutting down servers positioned on the extension?
3. Always happening at the same time?

But your drawing gives me sensations, definitely.

 

[Bjorn Smith]

No logs in the ups?
Extension is stupid i suppose?
No shutdown time configured in Bios?

I never looked in the ups, since I assumed that since all the servers was connected to the same outlet from the UPS, it would be all or none of the servers that would shut down.

No shutdown configured, since these are supposed to run 24/7

And I agree that an extension is kind of stupid

Questions would be, ...

1. What UPS & extension are you using?
2. Where are the shutting down servers positioned on the extension?
3. Always happening at the same time?

But your drawing gives me sensations, definitely.

1. Its a APC SmartUPS 2200i - so plenty of power and batteries report 100% - extension is just a "normal" extension cable with grounding where I converted the end to go into an UPS. (Not the cheapest - but not extremely expensive)
2. I haven't checked that - but when it happens next time - I will note what servers are connected where
3. Yes exactly the same time (within seconds - possibly because of time shift between servers)

 

[Rand__]

-If all 5 servers are on the same extension it cant be the UPS
-It being a stupid power strip (pun intended) I very much doubt its 'capable of shutting down individual outlets (unless they're switched)? - So i doubt its the extension either.
-Could you have a power surge? Are the first three outlets impacted? Theoretically the UPS should filter but... Check the UPS logs if you have a RMC. Why not if you dont?

Check IPMI system times if they differ, if yes then all might have gone down at the same, if no, then it sounds like a cascaded shutdown.
U sure you dont have a script that does remote power down? Maybe the message in the log is crap/incomplete?

 

[Bjorn Smith]

-If all 5 servers are on the same extension it cant be the UPS
-It being a stupid power strip (pun intended) I very much doubt its 'capable of shutting down individual outlets (unless they're switched)? - So i doubt its the extension either.

That was my initial thoughts as well

-Could you have a power surge? Are the first three outlets impacted? Theoretically the UPS should filter but... Check the UPS logs if you have a RMC.

I do have a network card for the UPS - so I will check the logs to see if I can see something. EDIT - just checked the logs and the logs were truncated :-( - I have bumped the size the UPS will store. A lot of useless logging of input power every 10 minutes.

U sure you dont have a script that does remote power down? Maybe the message in the log is crap/incomplete?

100% certain - I have never used stuff like that - I like my servers running 24/7 sucking my wallet dry :-D

 

[Rand__]

You have a cleaning lady? maybe she needed some outlets
Kids, a cat or in-laws who wanted to prank you?

 

[Bjorn Smith]

You have a cleaning lady? maybe she needed some outlets
Kids, a cat or in-laws who wanted to prank you?

No - not a possibility - my servers are on a shelf where the outlets are hidden - but good suggestion
I think I have to wait and see what happens next - and if next time my UPS has a log.

 

[gb00s]

Or all 5 went down and just 2 came up due to diff Bios settings handling power loss differently. Just smthg to check.

 

[Bjorn Smith]

Or all 5 went down and just 2 came up due to diff Bios settings handling power loss differently. Just smthg to check.

I checked that - and the two servers that stayed on did not have the "Power lost" event - but it might be a way to alleviate it if it happens again, i.e. make the servers start automatically.

 

[gb00s]

I checked that - and the two servers that stayed on did not have the "Power lost" event - but it might be a way to alleviate it if it happens again, i.e. make the servers start automatically.

Have these two servers the same bios settings in regards to logging? You should also see any shutdown event in /var/log/messages. You could also try:

last -x | less

to see all previous shutdown/reboot events.

 

[Bjorn Smith]

Have these two servers the same bios settings in regards to logging? You should also see any shutdown event in /var/log/messages. You could also try:

to see all previous shutdown/reboot events.

Servers all had mixed settings for "Restore power" - and some of the servers that shut down even had


My guess is that the server OS itself have 0 logging since its definetely a "lost power" scenario - not a shutdown.

Also server states:


So the loss is external from the server - or a failing PSU - but that seems strange that 3 servers should fail on the exact same time - two times in a row.

 

[Bjorn Smith]

I think my UPS is dying - I just lost power on 3 servers again - and now my UPS is beeping with "Please plug in battery" - so I guess I need to unplug my servers and get a new UPS.
Thanks for all the suggestions - it turned out to not be the Fujitsu servers

 

[gb00s]

Same time? Strange, but calibrate first. If batteries are ok your load shall not drop after calibration. I had the same issue with an UPS complaining as "Please plug in battery" because I was overloading it while running several compiling jobs on connected servers or through a significant power surge.

Manual for troubleshooting

 

[Bjorn Smith]

Same time? Strange, but calibrate first. If batteries are ok your load shall not drop after calibration. I had the same issue with an UPS complaining as "Please plug in battery" because I was overloading it while running several compiling jobs on connected servers or through a significant power surge.

Manual for troubleshooting

I am definetely not overloading the UPS - its a 2200VA rated UPS and I think the MAX load I can put on it with the servers I have running is <1000W.
Whe my servers are running normally its running around 10% capacity.

Luckily I had a PDU I could plug in, so I can re-use the plugs from the UPS - but I think I don't trust this UPS anymore - feel free to pick it up - it only weighs like 50+KG

The problem is getting rid of it - I doubt anyone would want to buy it - or pick it up for free - all calibrations that it does and self tests has been OK - so either something is broken inside - or it has just gotten old.

 

[Rand__]

And you know u can replace the battery on those, do you? Even just replace the "cells" (=smaller batteries) in the bigger one - no need to toss the whole UPS.
But calibrating is a good idea, followed by a test and potentially a load test

But it might be the cause, if the provided output exceeded the max load it could sustain (due to damaged/depleted battery ).
O/c it also could be defective as those on-line UPS convert utility power all the time (to clean up spikes), though I dont think it always involves the battery.
U could take out the battery to see if thats the problem or the UPS itself..

(Edited)

 

[Bjorn Smith]

But calibrating is a good idea, followed by a test and potentially a load test

Too much effort - my "lab" is homeprod - so I need it running 24/7 without me having to diagnose stuff like this.

I'm this close to just tossing all my servers in the bin and go back to a single server with everything and dramatically scale down so I don't have multiple switches, servers etc less complicated - although I have been happy with the resilience of ceph considering how many times I have lost power to my servers.

 

[Rand__]

You need homeprod, homeintegration, and hometest

Nah kidding, if you want 24/7 uptime then redundancy is key. If best effort is fine, then single box (and maybe a few spare parts or replacement box) suffices