0009234: test_pq memory corruption - MantisBT

ID	Project	Category	View Status	Date Submitted	Last Update

0009234	GNUnet	postgres library	public	2024-09-30 13:49	2024-10-10 10:20

Reporter	schanzen	Assigned To	schanzen
Priority	normal	Severity	minor	Reproducibility	have not tried
Status	closed	Resolution	fixed
Product Version	Git master
Target Version	0.22.1

Summary	0009234: test_pq memory corruption
Description	pq/pq_result_helper.c seems to have an issue in combination with postgres 16.3 (and earlier or later IDK). I tested GNUnet 0.21.0 and 0.22.0 and master with the same results. Running on latest Fedora.
Steps To Reproduce	MALLOC_PERTURB_=116 ./test_pq
Additional Information	valgrind: ==2365756== Memcheck, a memory error detector ==2365756== Copyright (C) 2002-2024, and GNU GPL'd, by Julian Seward et al. ==2365756== Using Valgrind-3.23.0 and LibVEX; rerun with -h for copyright info ==2365756== Command: test_pq ==2365756== 2024-09-30T13:48:03.505839+0200 test-pq-2365756 INFO Change in PQ event FD to -1 2024-09-30T13:48:03.531571+0200 test-pq-2365756 INFO New poll FD is -1 2024-09-30T13:48:03.636742+0200 test-pq-2365756 INFO Change in PQ event FD to 3 2024-09-30T13:48:03.636872+0200 test-pq-2365756 INFO New poll FD is 3 ==2365756== Invalid write of size 8 ==2365756== at 0x490DFEA: qconv_array (pq_query_helper.c:829) ==2365756== by 0x4907AE3: GNUNET_PQ_exec_prepared (pq.c:69) ==2365756== by 0x403F92: run_queries (test_pq.c:290) ==2365756== by 0x405B0D: main (test_pq.c:551) ==2365756== Address 0x59289f8 is 40 bytes inside a block of size 45 alloc'd ==2365756== at 0x4843866: malloc (vg_replace_malloc.c:446) ==2365756== by 0x487AA23: GNUNET_xmalloc_unchecked_ (common_allocation.c:164) ==2365756== by 0x487A404: GNUNET_xmalloc_ (common_allocation.c:59) ==2365756== by 0x490DED0: qconv_array (pq_query_helper.c:802) ==2365756== by 0x4907AE3: GNUNET_PQ_exec_prepared (pq.c:69) ==2365756== by 0x403F92: run_queries (test_pq.c:290) ==2365756== by 0x405B0D: main (test_pq.c:551) ==2365756== ==2365756== Syscall param socketcall.sendto(msg) points to uninitialised byte(s) ==2365756== at 0x4AAD8BD: send (in /usr/lib64/libc.so.6) ==2365756== by 0x4954890: ??? (in /usr/lib64/libpq.so.5.16) ==2365756== by 0x49582EC: ??? (in /usr/lib64/libpq.so.5.16) ==2365756== by 0x49598D8: ??? (in /usr/lib64/libpq.so.5.16) ==2365756== by 0x4959A26: PQsendQueryPrepared (in /usr/lib64/libpq.so.5.16) ==2365756== by 0x495DEC3: PQexecPrepared (in /usr/lib64/libpq.so.5.16) ==2365756== by 0x4907C50: GNUNET_PQ_exec_prepared (pq.c:92) ==2365756== by 0x403F92: run_queries (test_pq.c:290) ==2365756== by 0x405B0D: main (test_pq.c:551) ==2365756== Address 0x58c9e47 is 359 bytes inside a block of size 16,384 alloc'd ==2365756== at 0x4843866: malloc (vg_replace_malloc.c:446) ==2365756== by 0x49487BF: ??? (in /usr/lib64/libpq.so.5.16) ==2365756== by 0x494F832: PQconnectStart (in /usr/lib64/libpq.so.5.16) ==2365756== by 0x494FAF1: PQconnectdb (in /usr/lib64/libpq.so.5.16) ==2365756== by 0x49092E0: GNUNET_PQ_reconnect (pq_connect.c:433) ==2365756== by 0x49084D0: GNUNET_PQ_connect2 (pq_connect.c:129) ==2365756== by 0x49082EF: GNUNET_PQ_connect (pq_connect.c:74) ==2365756== by 0x40594D: main (test_pq.c:525) ==2365756== ==2365756== Conditional jump or move depends on uninitialised value(s) ==2365756== at 0x49153B5: extract_array_generic (pq_result_helper.c:1468) ==2365756== by 0x4907ED3: GNUNET_PQ_extract_result (pq.c:163) ==2365756== by 0x404163: run_queries (test_pq.c:319) ==2365756== by 0x405B0D: main (test_pq.c:551) ==2365756== ==2365756== Conditional jump or move depends on uninitialised value(s) ==2365756== at 0x491548D: extract_array_generic (pq_result_helper.c:1475) ==2365756== by 0x4907ED3: GNUNET_PQ_extract_result (pq.c:163) ==2365756== by 0x404163: run_queries (test_pq.c:319) ==2365756== by 0x405B0D: main (test_pq.c:551) ==2365756== ==2365756== Conditional jump or move depends on uninitialised value(s) ==2365756== at 0x487A361: GNUNET_xmalloc_ (common_allocation.c:56) ==2365756== by 0x491554A: extract_array_generic (pq_result_helper.c:1476) ==2365756== by 0x4907ED3: GNUNET_PQ_extract_result (pq.c:163) ==2365756== by 0x404163: run_queries (test_pq.c:319) ==2365756== by 0x405B0D: main (test_pq.c:551) ==2365756== ==2365756== Conditional jump or move depends on uninitialised value(s) ==2365756== at 0x4843810: malloc (vg_replace_malloc.c:446) ==2365756== by 0x487AA23: GNUNET_xmalloc_unchecked_ (common_allocation.c:164) ==2365756== by 0x487A404: GNUNET_xmalloc_ (common_allocation.c:59) ==2365756== by 0x491554A: extract_array_generic (pq_result_helper.c:1476) ==2365756== by 0x4907ED3: GNUNET_PQ_extract_result (pq.c:163) ==2365756== by 0x404163: run_queries (test_pq.c:319) ==2365756== by 0x405B0D: main (test_pq.c:551) ==2365756== ==2365756== Conditional jump or move depends on uninitialised value(s) ==2365756== at 0x48516C4: memset (vg_replace_strmem.c:1390) ==2365756== by 0x487AA4A: GNUNET_xmalloc_unchecked_ (common_allocation.c:167) ==2365756== by 0x487A404: GNUNET_xmalloc_ (common_allocation.c:59) ==2365756== by 0x491554A: extract_array_generic (pq_result_helper.c:1476) ==2365756== by 0x4907ED3: GNUNET_PQ_extract_result (pq.c:163) ==2365756== by 0x404163: run_queries (test_pq.c:319) ==2365756== by 0x405B0D: main (test_pq.c:551) ==2365756== ==2365756== Conditional jump or move depends on uninitialised value(s) ==2365756== at 0x4851712: memset (vg_replace_strmem.c:1390) ==2365756== by 0x487AA4A: GNUNET_xmalloc_unchecked_ (common_allocation.c:167) ==2365756== by 0x487A404: GNUNET_xmalloc_ (common_allocation.c:59) ==2365756== by 0x491554A: extract_array_generic (pq_result_helper.c:1476) ==2365756== by 0x4907ED3: GNUNET_PQ_extract_result (pq.c:163) ==2365756== by 0x404163: run_queries (test_pq.c:319) ==2365756== by 0x405B0D: main (test_pq.c:551) ==2365756== ==2365756== Conditional jump or move depends on uninitialised value(s) ==2365756== at 0x4851724: memset (vg_replace_strmem.c:1390) ==2365756== by 0x487AA4A: GNUNET_xmalloc_unchecked_ (common_allocation.c:167) ==2365756== by 0x487A404: GNUNET_xmalloc_ (common_allocation.c:59) ==2365756== by 0x491554A: extract_array_generic (pq_result_helper.c:1476) ==2365756== by 0x4907ED3: GNUNET_PQ_extract_result (pq.c:163) ==2365756== by 0x404163: run_queries (test_pq.c:319) ==2365756== by 0x405B0D: main (test_pq.c:551) ==2365756== ==2365756== Use of uninitialised value of size 8 ==2365756== at 0x485172E: memset (vg_replace_strmem.c:1390) ==2365756== by 0x487AA4A: GNUNET_xmalloc_unchecked_ (common_allocation.c:167) ==2365756== by 0x487A404: GNUNET_xmalloc_ (common_allocation.c:59) ==2365756== by 0x491554A: extract_array_generic (pq_result_helper.c:1476) ==2365756== by 0x4907ED3: GNUNET_PQ_extract_result (pq.c:163) ==2365756== by 0x404163: run_queries (test_pq.c:319) ==2365756== by 0x405B0D: main (test_pq.c:551) ==2365756== ==2365756== Conditional jump or move depends on uninitialised value(s) ==2365756== at 0x4851744: memset (vg_replace_strmem.c:1390) ==2365756== by 0x487AA4A: GNUNET_xmalloc_unchecked_ (common_allocation.c:167) ==2365756== by 0x487A404: GNUNET_xmalloc_ (common_allocation.c:59) ==2365756== by 0x491554A: extract_array_generic (pq_result_helper.c:1476) ==2365756== by 0x4907ED3: GNUNET_PQ_extract_result (pq.c:163) ==2365756== by 0x404163: run_queries (test_pq.c:319) ==2365756== by 0x405B0D: main (test_pq.c:551) ==2365756== ==2365756== Conditional jump or move depends on uninitialised value(s) ==2365756== at 0x4851769: memset (vg_replace_strmem.c:1390) ==2365756== by 0x487AA4A: GNUNET_xmalloc_unchecked_ (common_allocation.c:167) ==2365756== by 0x487A404: GNUNET_xmalloc_ (common_allocation.c:59) ==2365756== by 0x491554A: extract_array_generic (pq_result_helper.c:1476) ==2365756== by 0x4907ED3: GNUNET_PQ_extract_result (pq.c:163) ==2365756== by 0x404163: run_queries (test_pq.c:319) ==2365756== by 0x405B0D: main (test_pq.c:551) ==2365756== ==2365756== Conditional jump or move depends on uninitialised value(s) ==2365756== at 0x4850E1E: bcmp (vg_replace_strmem.c:1233) ==2365756== by 0x404328: run_queries (test_pq.c:325) ==2365756== by 0x405B0D: main (test_pq.c:551) ==2365756== 2024-09-30T13:48:04.123546+0200 test-pq-2365756 INFO got oid 18630 for type foo 2024-09-30T13:48:04.163772+0200 test-pq-2365756 INFO Starting event scheduler 2024-09-30T13:48:04.164037+0200 test-pq-2365756 INFO New poll FD is 3 2024-09-30T13:48:04.164276+0200 test-pq-2365756 INFO Activating poll job on 3 2024-09-30T13:48:04.167110+0200 test-pq-2365756 INFO Executing PQ command `LISTEN XWFP422M1AM9FV94GGP20NJY0MPZRMF0QFVNTVP1F8JRDBNSKY0ZG' 2024-09-30T13:48:04.168419+0200 test-pq-2365756 INFO Change in PQ event FD to -1 2024-09-30T13:48:04.168490+0200 test-pq-2365756 INFO New poll FD is -1 2024-09-30T13:48:04.170162+0200 test-pq-2365756 INFO Executing PQ command `LISTEN XWFP422M1AM9FV94GGP20NJY0MPZRMF0QFVNTVP1F8JRDBNSKY0ZG' 2024-09-30T13:48:04.181250+0200 test-pq-2365756 INFO Change in PQ event FD to 3 2024-09-30T13:48:04.181384+0200 test-pq-2365756 INFO New poll FD is 3 2024-09-30T13:48:04.181506+0200 test-pq-2365756 INFO Activating poll job on 3 2024-09-30T13:48:04.181628+0200 test-pq-2365756 INFO Executing PQ command `LISTEN XWFP422M1AM9FV94GGP20NJY0MPZRMF0QFVNTVP1F8JRDBNSKY0ZG' 2024-09-30T13:48:04.183652+0200 test-pq-2365756 INFO Executing command `NOTIFY XWFP422M1AM9FV94GGP20NJY0MPZRMF0QFVNTVP1F8JRDBNSKY0ZG, 'D1JPRV3F'' 2024-09-30T13:48:04.185368+0200 test-pq-2365756 INFO PG poll job active 2024-09-30T13:48:04.190565+0200 test-pq-2365756 INFO Received notification xwfp422m1am9fv94ggp20njy0mpzrmf0qfvntvp1f8jrdbnsky0zg with extra data `hello' 2024-09-30T13:48:04.193359+0200 test-pq-2365756 INFO PG poll job finishes after 1 events 2024-09-30T13:48:04.194684+0200 test-pq-2365756 INFO Executing PQ command `UNLISTEN XWFP422M1AM9FV94GGP20NJY0MPZRMF0QFVNTVP1F8JRDBNSKY0ZG' 2024-09-30T13:48:04.195236+0200 test-pq-2365756 INFO Stopping PQ event scheduler job ==2365756== ==2365756== HEAP SUMMARY: ==2365756== in use at exit: 3,808 bytes in 18 blocks ==2365756== total heap usage: 2,447 allocs, 2,429 frees, 370,719 bytes allocated ==2365756== ==2365756== LEAK SUMMARY: ==2365756== definitely lost: 216 bytes in 1 blocks ==2365756== indirectly lost: 2,048 bytes in 1 blocks ==2365756== possibly lost: 0 bytes in 0 blocks ==2365756== still reachable: 1,544 bytes in 16 blocks ==2365756== suppressed: 0 bytes in 0 blocks ==2365756== Rerun with --leak-check=full to see details of leaked memory ==2365756== ==2365756== Use --track-origins=yes to see where uninitialised values come from ==2365756== For lists of detected and suppressed errors, rerun with: -s ==2365756== ERROR SUMMARY: 47 errors from 13 contexts (suppressed: 0 from 0)
Tags	No tags attached.

schanzen 2024-09-30 15:46 administrator ~0023407	One thing certainly not portable was https://git.gnunet.org/gnunet.git/commit/?id=2825a3c6f54546226e6448df2ac53f00fceb658d sizeof (size_t) is not portable and caused the first valgring errors

schanzen 2024-09-30 16:37 administrator ~0023408	Fix committed to master branch.

schanzen 2024-09-30 16:39 administrator ~0023409	Two issues still remain: ==2397690== Conditional jump or move depends on uninitialised value(s) ==2397690== at 0x4850E1E: bcmp (vg_replace_strmem.c:1233) ==2397690== by 0x404328: run_queries (test_pq.c:325) ==2397690== by 0x405B0D: main (test_pq.c:551) ==2397690== In the code, the assertion is based on hash codes where "hc" is not initialized at all. So the assertion should probably always fail. The issue right at the top of the valigrind output: ==2397690== Syscall param socketcall.sendto(msg) points to uninitialised byte(s) ==2397690== at 0x4AAD8BD: send (in /usr/lib64/libc.so.6) ==2397690== by 0x4954890: ??? (in /usr/lib64/libpq.so.5.16) ==2397690== by 0x49582EC: ??? (in /usr/lib64/libpq.so.5.16) ==2397690== by 0x49598D8: ??? (in /usr/lib64/libpq.so.5.16) ==2397690== by 0x4959A26: PQsendQueryPrepared (in /usr/lib64/libpq.so.5.16) ==2397690== by 0x495DEC3: PQexecPrepared (in /usr/lib64/libpq.so.5.16) ==2397690== by 0x4907C60: GNUNET_PQ_exec_prepared (pq.c:92) ==2397690== by 0x403F92: run_queries (test_pq.c:290) ==2397690== by 0x405B0D: main (test_pq.c:551) ==2397690== Address 0x58c9e47 is 359 bytes inside a block of size 16,384 alloc'd ==2397690== at 0x4843866: malloc (vg_replace_malloc.c:446) ==2397690== by 0x49487BF: ??? (in /usr/lib64/libpq.so.5.16) ==2397690== by 0x494F832: PQconnectStart (in /usr/lib64/libpq.so.5.16) ==2397690== by 0x494FAF1: PQconnectdb (in /usr/lib64/libpq.so.5.16) ==2397690== by 0x49092F0: GNUNET_PQ_reconnect (pq_connect.c:433) ==2397690== by 0x49084E0: GNUNET_PQ_connect2 (pq_connect.c:129) ==2397690== by 0x49082FF: GNUNET_PQ_connect (pq_connect.c:74) ==2397690== by 0x40594D: main (test_pq.c:525) seems to be in the guts of libpq? IDK

schanzen 2024-09-30 19:51 administrator ~0023410	Fix committed to master branch.

schanzen 2024-10-10 10:20 administrator ~0023497	released

Date Modified	Username	Field	Change
2024-09-30 13:49	schanzen	New Issue
2024-09-30 13:49	schanzen	Status	new => assigned
2024-09-30 13:49	schanzen	Assigned To	=> oec
2024-09-30 15:46	schanzen	Note Added: 0023407
2024-09-30 16:37	schanzen	Changeset attached	=> gnunet master 67c3a572
2024-09-30 16:37	schanzen	Note Added: 0023408
2024-09-30 16:37	schanzen	Assigned To	oec => schanzen
2024-09-30 16:37	schanzen	Status	assigned => resolved
2024-09-30 16:37	schanzen	Resolution	open => fixed
2024-09-30 16:39	schanzen	Status	resolved => confirmed
2024-09-30 16:39	schanzen	Note Added: 0023409
2024-09-30 16:39	schanzen	Assigned To	schanzen => oec
2024-09-30 19:51	schanzen	Changeset attached	=> gnunet master 1c0cf5bf
2024-09-30 19:51	schanzen	Note Added: 0023410
2024-09-30 19:51	schanzen	Assigned To	oec => schanzen
2024-09-30 19:51	schanzen	Status	confirmed => resolved
2024-10-10 10:20	schanzen	Status	resolved => closed
2024-10-10 10:20	schanzen	Note Added: 0023497