Hopefully someone here has wrestled with serial server oddities and can shed some light on this...
I've got a serial console server made by Digi (TS8 PortServer) setup in a fairly vanilla mode: 9600-8-N-1....telnet to port 500X gets you to port X. Setup for a vt100 terminal type. Other VTs I tried didn't seem to make a difference.
Problem is when attached to a Cisco switch I had laying around I get seemily random garble output when accessing the console of a remote Cisco device. (e.g. "show run" will get a few garbled lines halfway through, Holding down Enter will produce some garbled text every few lines).
When attached to a Juniper device everything appears normal. The problem follows the port if I swap the Cisco device to the port the J-box was on.
Other issues I've noticed..cannot use arrow keys to search command buffer.
My Google-fu is failing me in coming up with the right name for the effect I'm seeing...
Can you configure the port (on the switch and the console server) for flow control? You might be experiencing an overflow issue if the CPU of the terminal server gets busy or a buffer gets full. Maybe RTS/CTS (if the cable has the pins) or even XON/XOFF (if it doesn't).
Other issues I've noticed..cannot use arrow keys to search command
This is going to be a tougher one. Might be a difference in character encoding. Here is the VT100 spec:
* ESC D cursor down - at bottom of region, scroll up
* ESC M cursor up - at top of region, scroll down
Arrows Standard Applications IBM Keypad
Up ESC [ A ESC O A Alt 9
Down ESC [ B ESC O B Alt 0
Right ESC [ C ESC O C Alt -
Left ESC [ D ESC O D Alt =
So you probably need to check your keyboard encoding. It likely differs from VT100 escape sequences.
Also, if you have several devices connected to that terminal server, see if you have one that is spewing debug or other information out the console port. That one might be causing some buffer overrun situations or keeping the CPU busy so it loses characters. Line noise can cause garbled data, too. But I would try flow control first. One thing I have seen before also is ground loops causing issues. Some serial devices actually tie signal ground to chassis ground. If you have a cable connecting two such devices and there is some ground potential difference, you can create a ground loop and introduce noise (and things like sparks, fire, blown fuses, etc.) if the ground potential difference is great enough between the two devices.