There is a system where our customer had the following error messages:
…
APTY Trying to allocate /dev/ptyr7
APTY Trying to allocate /dev/ptyr8
APTY Trying to allocate /dev/ptyr9
…
The background is, that it is a custom application, which is used from ServiceGuard on a two-node cluster.
I guess it is around ~15 years old. We must migrate this package because of the leasings end of the server.
After we moved this package onto another two-node cluster, we’ve got the error message.
The HW config is identical, and we needed to set the kernel parameters, especially npty should have been raised to 1200:
# kmtune -q npty
Parameter Current Dyn Planned Module Version
===============================================================================
npty 1200 - 1200
#
But the error remained. We opened a call at the HP Support, and they gave us a tool that displays a summary about BSD-style ptys. This is how it works:
# ./tioshow -bptys
tioshow (1.7.7)
Note: HP CONFIDENTIAL
libp4 (9.318): Opening /stand/vmunix /dev/kmem
Loading symbols from /stand/vmunix
Kernel TEXT pages not requested in crashconf
Will use an artificial mapping from a.out TEXT pages
Loading symbols from /stand/dlkm/mod.d/rng
Note: Consider P4_ELF_WARNING=2 to get more details
Note: No debug information for this module
Loading symbols from /stand/dlkm/mod.d/ipf
Loading symbols from /stand/dlkm/mod.d/pfil
tioshow (1.7.7) output
=====================
= Table Of Contents =
=====================
* BSD ptys
============
= BSD ptys =
============
=======================
= Summary of BSD ptys =
=======================
master slave(s)
dev file pid p_comm pid p_comm
———- ————– —– —— —– ——
0×10000000 /dev/ptyp0 913 ptydaemon 0
0×11000001 /dev/ttyp1 6577 xterm 6709 ksh
6960 foei
0×11000002 /dev/ttyp2 7080 xterm 7083 ksh
7114 fpuj
18927 sh
18929 krafpul
18931 krafpul
18932 java
0×11000003 /dev/ttyp3 20238 xterm 20240 ksh
20272 sbal
0×11000004 /dev/ttyp4 10262 xterm 10263 ksh
10328 voad
0×11000005 /dev/ttyp5 24625 xterm 24626 ksh
24911 foau
0×11000006 /dev/ttyp6 7593 xterm 7594 ksh
7625 bevn
0×11000007 /dev/ttyp7 0 0
0×11000008 /dev/ttyp8 13517 xterm 13518 ksh
13549 eins
0×11000009 /dev/ttyp9 13589 xterm 13590 ksh
13623 rtel
0x1100000a /dev/ttypa 13663 xterm 13664 ksh
13695 foau
9388 sh
9390 krafpul
9392 krafpul
9393 java
0x1100000b /dev/ttypb 13715 xterm 13716 ksh
13785 labd
0x1100000c /dev/ttypc 14393 xterm 14396 ksh
14427 foei
0x1100000d /dev/ttypd 14468 xterm 14469 ksh
14504 fpuj
0x1100000e /dev/ttype 20316 xterm 20317 ksh
20350 mimenue
0x1100000f /dev/ttypf 21020 xterm 21043 ksh
21192 reih
0×11000010 /dev/ttyq0 14908 xterm 14912 ksh
15011 bevn
0×11000011 /dev/ttyq1 23283 xterm 23284 ksh
23315 sbar
0×11000012 /dev/ttyq2 15205 xterm 15206 ksh
15255 warv
0×11000013 /dev/ttyq3 0 0
0x110000b0 /dev/pty/ttya0 4317 netptyd 0
0x110000b1 /dev/pty/ttya1 4301 netptyd 0
0x110000b2 /dev/pty/ttya2 4305 netptyd 0
0x110000b3 /dev/pty/ttya3 4313 netptyd 0
0x110000b4 /dev/pty/ttya4 4293 netptyd 0
0x110000b5 /dev/pty/ttya5 4297 netptyd 0
0x110000b6 /dev/pty/ttya6 4285 netptyd 0
0x110000b7 /dev/pty/ttya7 4309 netptyd 0
0x110000b8 /dev/pty/ttya8 4321 netptyd 0
0x110000b9 /dev/pty/ttya9 4325 netptyd 0
0x110000ba /dev/pty/ttyaa 4329 netptyd 0
0x110000bb /dev/pty/ttyab 4289 netptyd 0
0x110000bc /dev/pty/ttyac 4333 netptyd 0
0x110000bd /dev/pty/ttyad 4337 netptyd 0
0x110000be /dev/pty/ttyae 4341 netptyd 0
0x110000bf /dev/pty/ttyaf 4345 netptyd 0
0x110000c0 /dev/pty/ttyb0 4349 netptyd 0
0x110000c1 /dev/pty/ttyb1 4353 netptyd 0
0x110000c2 /dev/pty/ttyb2 4613 netptyd 0
0x110000c3 /dev/pty/ttyb3 4625 netptyd 0
0x110000c4 /dev/pty/ttyb4 4617 netptyd 0
0x110000c5 /dev/pty/ttyb5 4629 netptyd 0
0x110000c6 /dev/pty/ttyb6 4633 netptyd 0
0x110000c7 /dev/pty/ttyb7 4641 netptyd 0
0x110000c8 /dev/pty/ttyb8 4621 netptyd 0
0x110000c9 /dev/pty/ttyb9 4609 netptyd 0
0x110000ca /dev/pty/ttyba 4637 netptyd 0
0x110000cb /dev/pty/ttybb 4649 netptyd 0
0x110000cc /dev/pty/ttybc 4645 netptyd 0
0x110000cd /dev/pty/ttybd 4605 netptyd 0
0x110000ce /dev/pty/ttybe 4657 netptyd 0
0x110000cf /dev/pty/ttybf 4653 netptyd 0
0x110000d0 /dev/pty/ttyc0 4601 netptyd 0
Configured BSD ptys (npty) = 1200
Active BSD ptys = 53
Available BSD ptys = 1147
End of tioshow output
#
After this it was identified that the application of our customer uses the device files directly from under /dev. I mean these:
# ll /dev/pty[pqr]*
crw-rw-rw- 2 bin tty 16 0×000000 Nov 15 2000 /dev/ptyp0
crw-rw-rw- 2 bin tty 16 0×000001 Nov 15 2000 /dev/ptyp1
crw-rw-rw- 2 bin tty 16 0×000002 Nov 15 2000 /dev/ptyp2
crw-rw-rw- 2 bin tty 16 0×000003 Nov 15 2000 /dev/ptyp3
crw-rw-rw- 2 bin tty 16 0×000004 Nov 15 2000 /dev/ptyp4
crw-rw-rw- 2 bin tty 16 0×000005 Nov 15 2000 /dev/ptyp5
crw-rw-rw- 2 bin tty 16 0×000006 Nov 15 2000 /dev/ptyp6
crw-rw-rw- 2 bin tty 16 0×000007 Nov 15 2000 /dev/ptyp7
crw-rw-rw- 2 bin tty 16 0×000008 Nov 15 2000 /dev/ptyp8
crw-rw-rw- 2 bin tty 16 0×000009 Nov 15 2000 /dev/ptyp9
crw-rw-rw- 2 bin tty 16 0x00000a Nov 15 2000 /dev/ptypa
crw-rw-rw- 2 bin tty 16 0x00000b Nov 15 2000 /dev/ptypb
crw-rw-rw- 2 bin tty 16 0x00000c Nov 15 2000 /dev/ptypc
crw-rw-rw- 2 bin tty 16 0x00000d Nov 15 2000 /dev/ptypd
crw-rw-rw- 2 bin tty 16 0x00000e Nov 15 2000 /dev/ptype
crw-rw-rw- 2 bin tty 16 0x00000f Nov 15 2000 /dev/ptypf
crw-rw-rw- 2 bin tty 16 0×000010 Nov 15 2000 /dev/ptyq0
crw-rw-rw- 2 bin tty 16 0×000011 Nov 15 2000 /dev/ptyq1
crw-rw-rw- 2 bin tty 16 0×000012 Nov 15 2000 /dev/ptyq2
crw-rw-rw- 2 bin tty 16 0×000013 Nov 15 2000 /dev/ptyq3
crw-rw-rw- 2 bin tty 16 0×000014 Nov 15 2000 /dev/ptyq4
crw-rw-rw- 2 bin tty 16 0×000015 Nov 15 2000 /dev/ptyq5
crw-rw-rw- 2 bin tty 16 0×000016 Nov 15 2000 /dev/ptyq6
crw-rw-rw- 2 bin tty 16 0×000017 Nov 15 2000 /dev/ptyq7
crw-rw-rw- 2 bin tty 16 0×000018 Nov 15 2000 /dev/ptyq8
crw-rw-rw- 2 bin tty 16 0×000019 Nov 15 2000 /dev/ptyq9
crw-rw-rw- 2 bin tty 16 0x00001a Nov 15 2000 /dev/ptyqa
crw-rw-rw- 2 bin tty 16 0x00001b Nov 15 2000 /dev/ptyqb
crw-rw-rw- 2 bin tty 16 0x00001c Nov 15 2000 /dev/ptyqc
crw-rw-rw- 2 bin tty 16 0x00001d Nov 15 2000 /dev/ptyqd
crw-rw-rw- 2 bin tty 16 0x00001e Nov 15 2000 /dev/ptyqe
crw-rw-rw- 2 bin tty 16 0x00001f Nov 15 2000 /dev/ptyqf
crw-rw-rw- 2 bin tty 16 0×000020 Nov 15 2000 /dev/ptyr0
crw-rw-rw- 2 bin tty 16 0×000021 Nov 15 2000 /dev/ptyr1
crw-rw-rw- 2 bin tty 16 0×000022 Nov 15 2000 /dev/ptyr2
crw-rw-rw- 2 bin tty 16 0×000023 Nov 15 2000 /dev/ptyr3
crw-rw-rw- 2 bin tty 16 0×000024 Nov 15 2000 /dev/ptyr4
crw-rw-rw- 2 bin tty 16 0×000025 Nov 15 2000 /dev/ptyr5
crw-rw-rw- 2 bin tty 16 0×000026 Nov 15 2000 /dev/ptyr6
crw-rw-rw- 2 bin tty 16 0×000027 Nov 15 2000 /dev/ptyr7
crw-rw-rw- 2 bin tty 16 0×000028 Nov 15 2000 /dev/ptyr8
crw-rw-rw- 2 bin tty 16 0×000029 Nov 15 2000 /dev/ptyr9
crw-rw-rw- 2 bin tty 16 0x00002a Nov 15 2000 /dev/ptyra
crw-rw-rw- 2 bin tty 16 0x00002b Nov 15 2000 /dev/ptyrb
crw-rw-rw- 2 bin tty 16 0x00002c Nov 15 2000 /dev/ptyrc
crw-rw-rw- 2 bin tty 16 0x00002d Nov 15 2000 /dev/ptyrd
crw-rw-rw- 2 bin tty 16 0x00002e Nov 15 2000 /dev/ptyre
crw-rw-rw- 2 bin tty 16 0x00002f Nov 15 2000 /dev/ptyrf
#
On a standard system there is only 48 pty devices under /dev, the rest were moved to /dev/ptym. So if you change the kernel parameter and after this you create those devices with insf, they will be created under /dev/ptym. These 48 in /dev were just left there for compatibility reasons. This is the response we’ve got from HP Support:
======================
The following is an explanation of hp-ux pty device files.
HP-UX put all the ptys and ttys into separate directories to try to decrease the size of /dev since in some cases, up to
27900 * 2 = 55,800 pty/tty files could exist! With so many files, /dev becomes somewhat unmanageable; the separation
helps to make the file system more manageable.
However, moving the ptys and ttys causes a problem: what happens to programs that are hard-coded to look in /dev for ptys and
ttys? For compatibility, HP-UX creates 48 ptys and 48 ttys in /dev so that those programs will still work. HP programs should
try to leave these available as long as possible to ensure that the non-compatible programs will be able to run
The customer application looks for pty devices under /dev.
I have checked the pty devices files from the old & new system and it has the same number of pty device files under /dev,
If you look both the systems have 48 pty devices under /dev , but the new system still needs more pty devices , so it gives the error “all ptys in use”.
So the new system may be having more users/applications which uses the pty devices under /dev.
So I would suggest to engage the application vendor to check their pty look/opening behaviour.
======================
As of we were beyond one’s power to contact the developer of the app, the solution/workaround was to create some hardlinks to the newly created devices under /dev. Alas the advice from HP was wrong, as they told us to create these with ‘ln -s’, but after all I realized that they should be hardlinked instead of symlinking. After this our problem with pty’s has solved. There were some other login-related problems but these were related to a decomissioned NIS domain. And I should say that after this pty issue the customer had a constant paranoia with ptys and he thought every issue is relating to this pty problem, every Remedy ticket has come with the following intro:
“PTY problem, please check!” :)
UPDATED: The link to the tioshow tool was removed. See comments.
2 Comments
That’s a very interesting post Viktor, Thanks. Concerning tioshow, while it can be useful I’m not sure you are allowed by HP to post it publicly. Anyone with a contract can normally open a case at the Response Center to request to get a copy of tioshow, the same with crashinfo, shminfo, etc. That’s just my opinion, I don’t work for HP.
You are right Olivier, I have removed the link.