Barnyard2 and MariaDB
-
You are correct. With the CFLAGS
-fpack-struct
enabled, Barnyard2 crashed immediately with signal 11 segmentation fault.I replaced it with
-mno-unaligned-access
. It crashed with signal 10 again. It seem to be a clang bug because the no unaligned access flag doesn't work.Any suggestion? I can't find the fix commit in snort's Makefile under your name. How can you address it?
-
@rickyzhang said in Barnyard2 and MariaDB:
This is very good pieces of educational reading. I read some alignment code in C struct in Linux kernel. But I never knew why until now. Thanks!
As you said, this memory access alignment problem can be mitigated by a compiler (Perhaps, it is not ideal to disable optimization).
No, disabling optimization is not ideal, but the choice was that or no Snort on the SG-3100 (or any other ARM-based appliance running pfSense). So we (me and the pfSense developer team) chose the "at least make it work" option.
The real fix for this is to never have the error within the source code to start with. This unaligned access problem almost always happens when "casting" a pointer variable from one type to another. But this is frequently required in C programming in order to stop the compiler from complaining about mismatched or wrong type function arguments. Going through C source code that you didn't write and trying to figure out where casting problems exist and then determine if fixing them will break something else or not is a tall order.
-
@rickyzhang said in Barnyard2 and MariaDB:
You are correct. With the CFLAGS
-fpack-struct
enabled, Barnyard2 crashed immediately with signal 11 segmentation fault.I replaced it with
-mno-unaligned-access
. It crashed with signal 10 again. It seem to be a clang bug because the no unaligned access flag doesn't work.Any suggestion? I can't find the fix commit in snort's Makefile under your name. How can you address it?
As you see, those flags are basically worthless because the compiler cannot reliably detect when unaligned access will occur - especially when the programmer is using nested casting. Also, on FreeBSD the actual compiler used is llvm. I actually disassembled the machine code and verified at least one place in Snort where the Signal 10 was happening. It is caused by the llvm compiler's choice of which of two register load instructions to use: LDR or LDM. LDR supports unaligned access while LDM does not. Same thing for STR and STM. Some details can be found in this post: https://medium.com/@iLevex/the-curious-case-of-unaligned-access-on-arm-5dd0ebe24965.
When optimizations are disabled, the llvm compiler will always use the LDR and STR instructions which do support unaligned access auto-fixup. With optimizations enabled, the compiler will choose to use the LDM and STM instructions instead. I saw this behavior during my Snort testing on the SG-3100 a couple of years ago.
The file we produced to take care of disabling optimizations when compiling for ARM is contained in the patch file
patch-pfSense-ARM317.diff
located in the files sub-directory of the Snort binary port. The patch adds additional code to theconfigure
script for the project which detects an ARM build and disables compiler optimizations. -
I admit I abused the CFLAGS
-fpack-struct
. I thought those struct flagged by clang struct padding warning comes from network protocol header for parsing the packet. It turns out they are not. Turning on packing made the matter worse.The CFLAGS
-mno-unaligned-access
supposed to do as the flag instructed by llvm/Clang (We are talking about the same thing). But it didn't do that intelligently.Note that SG-3100 is ARMv7 architecture. But we compile binary in ARMv6. In your quoted medium article, it stated that:
Beginning with ARMv7, however, unaligned access began to be supported. It now does the expected, i.e. breaking up the access into multiple smaller reads and builds up the value as a “traditional” x86 CPU would do it.
In FreeBSD 12.0, it starts to support ARMv7. We can actually build ARMv7 binary. Perhaps, that can mitigate your pain to patch.
I wonder when you said "nested casting" in your previous thread. Do you refer to int* pointer cast to char* pointer and you access memory through char* pointer cause memory misaligned access?
Thanks!
-
The documentation is sort of misleading. True, the ARMv7 chips can and will do the auto-fixup of unaligned access, but NOT when the LDM or STM opcodes are used. Those opcodes never support unaligned access. The llvm compiler will use those opcodes when optimizing the generated binary code because they are faster than their counterparts (LDR and STR) which DO support unaligned access. That's why we fixed it in the Snort and Suricata binaries the way we did. You have to get the compiler to not use those problematic opcodes. You can read more about that here: http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.faqs/ka15414.html.
Your example of defining a pointer as an integer pointer (int*) and then casting it to a char* is one method of causing unaligned access to happen. This can also happen when you are casting struct pointers (struct*) as well. By "nesting" I am using my own definition of casting a cast, or in other words wrapping multiple casts inside nested parentheses. Not a great piece of coding, but you will find it frequently in C source code.
You need to be versed in the internal microcode sequencing of a CPU to fully understand all about unaligned access. Back in my olden days I went to a school on an industrial mainframe where we actually walked through and had to learn in class exactly how the flip-flops and logic gates were wired together to produce shift registers, adders and address decoders to produce the CPU of the computer. The circuit boards of that mainframe were actually constructed using perf-boards with wirewrap connections (no printed circuits at all). The CPU of this mainframe was actually about a dozen wirewrapped circuit boards all connected via a backplane. This taught me in great detail how modern CPUs actually work. In fact, I could troubleshoot that processor and fix it by replacing a faulty DTL (predecessor to TTL logic) chip in a socket. The hardware was certainly primitive by today's standards, but learning in exquisite detail how it all worked to shift bits around and move them to and from memory (which was a whopping 16K magnetic core, by the way) was very helpful. I've built on that training ever since then.
-
For sure, the more you know, the better off you will be.
But the cornerstone of computer science is abstraction and isolation. The less you depend on, the more portable your code is. The things like memory alignment access which relates to target platform code generation should be handled by compiler. So I'd expect in ARMv6 or later architecture when compiler is not sure if memory misalign access happen or not, it should use LDR/STR by default. But if the compiler knows memory access is aligned in sure case, it can use LDM/STM to do multiple registers load/store to speed things up. IMO, it is compiler's fault to do early optimization to duck this up. Premature optimization is the root of all evil.
I can NOT find any document from clang regarding to their CFLAGS
-mno-unaligned-access
but I did find the doc in GCC. clang wants to be compatible with GCC. I think this might be a bug in clang which does not fulfill the promise.BTW, the medium blog is misleading, according the ARM document. I don't like those leftist blogging from medium in general.
OK. I stop my ranting. Now I want to know how to enable core dump in FreeBSD. I double check parameters from sysctl:
[2.4.4-RELEASE][admin@pfSense.localdomain]/root/.ssh: sysctl -a | grep core kern.corefile: /root/%N.core kern.coredump_devctl: 0 kern.nodump_coredump: 0 kern.coredump: 1 kern.capmode_coredump: 0 kern.sugid_coredump: 0 kern.coredump_pack_vmmapinfo: 1 kern.coredump_pack_fileinfo: 1 debug.ncores: 5 debug.elf32_legacy_coredump: 0
The
kern.coredump
has been enabled and/root
folder is R/W by root. But I still doesn't see core dump file. I'm scratching my head. I want to know exactly where cause bus error. I'm sure it is not the struct padding. I read those code in Qt Creator IDE which use clang to flag struct padding. The padding happen only in the last struct member, which is harmless in parsing protocol header.Thanks!
-
I found this information relative to the
-mno-unaligned-access
option for clang/llvm:https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=207096
Where the take-home message for me was in Comment 4:While options like -mno-unaligned-access will make the compiled code
avoid adding new misalignments as "optimizations" when the original
code does not misalign it will not repair code that directly generates
misalignments. (The alignment fixes to clang++ were all source code
fixes, not compiler option changes.)So that option does not appear to be there to "fix" source code. It's just there to make sure the compiler itself does not generate an unaligned access operation while optimizing.
-
I got that both code and compiler needs to work hand in hand to address memory misaligned access.
I found this article from Linux kernel development: https://www.kernel.org/doc/Documentation/unaligned-memory-access.txt
There two different types of memory misaligned access source in C code:
- struct member.
- pointer casting.
I have confident that Barnyard2 is free from the 1st source. I use Qt Creator to flag struct padding. Only two are flagged. They are flagged because of the last member need to be padded in the end. It seems harmless. I also eyeball 10 of random struct by grep
typedef struct
. They looks OK to me.So it might be pointer casting. I think I need the core dump to find them out and fix it. I have no better idea how to get the core dump in FreeBSD. It seem that it didn't work as expected in my SG-3100. Did pfsense customize kernel?
-
I found three possible ways to find the misaligned memory access from Barnyard2 source code:
-
Us the CFLAGS ‘’’-Wcast-align’’’ to flag potential ones. Hope there are few false positives.
-
Create a XCode project and let XCode’s clang does flagging. I have done several C porting to iOS. But I have never encountered the misaligned memory access. See: https://developer.apple.com/documentation/code_diagnostics/undefined_behavior_sanitizer/misaligned_pointer#overview
-
Run the binary and get the core dump like waiting a random bomb exploding.
-
-
@rickyzhang said in Barnyard2 and MariaDB:
I found three possible ways to find the misaligned memory access from Barnyard2 source code:
-
Us the CFLAGS ‘’’-Wcast-align’’’ to flag potential ones. Hope there are few false positives.
-
Create a XCode project and let XCode’s clang does flagging. I have done several C porting to iOS. But I have never encountered the misaligned memory access. See: https://developer.apple.com/documentation/code_diagnostics/undefined_behavior_sanitizer/misaligned_pointer#overview
-
Run the binary and get the core dump like waiting a random bomb exploding.
I suspect there are many places in the source code where it could happen. And the hard thing about unaligned access is that it can be sort of random as it can be triggered when certain data combinations happen in some piece of code. For example, consider an "if-then" test that is rarely "true", but when it is "true" the code executed within the "then" portion triggers an unaligned access. Makes finding all the bugs very hard since they don't reliably reproduce.
I have an email relationship with the Snort developer team, and I have corresponded with them about the Snort issues on ARM hardware. They acknowledged that the Snort 2.9.x source code has a number of potential bug spots with unaligned access. I suspect Barnyard2 may have many as well.
My suggestion is to turn off compiler optimizations so that clang/llvm will not use the problematic LDM/STM opcodes. Yes, the code might be a teeny tiny bit slower, but at least it will work. Then concentrate on fixing the other SQL bugs you started out working on.
-
-
First of all, SQL syntax bug has been patched. Bus error is a road block now.
There are 169 pointer pointer casting places got flagged by clang's
-Wcast-align
analysis. It is a lot as you expected.I take a close look into the first one
sf_iph.c:132:16: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'IP6Frag *' (aka 'struct _IP6Frag *') increases required alignment from 1 to 4
.The pointer casting looks normal/common use case when you cast a pointer to a generic buffer into a pointer to a struct .
I first tried
--pointer_alignment=1
compiler flag from ARM document. Ideally, it forces the compiler to treat all pointer casting is unaligned. But unfortunately clang doesn't support this compiler option. It failed to compile.Then, I tried to add
__packed
qualifier in pointer casting. It can compile but I'm not sure if it takes effect or not. It still flagged this casting is unaligned by-Wcast-align
.For now, I give it up. I will try your approach to disable optimization tomorrow. You added a architecture check in configure and an option(?) optimization=NO for ARM. It looks like specific to snort build rather than a generic fix for others. Should I use
-O0
flag or debug flag-g
?Thanks!
See
-Wcast-align
log:132 frag_hdr = (IP6Frag*)p->ip6_extensions[p->ip6_frag_index].data;
Ricky@gtx-freebsd ~ $ grep "\[\-Wcast\-align\]" /usr/local/poudriere/data/logs/bulk/pfsense-port-11-2-armv6-pfsense-v2-4-4-3/latest/logs/barnyard2-1.13_1.log sf_iph.c:132:16: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'IP6Frag *' (aka 'struct _IP6Frag *') increases required alignment from 1 to 4 [-Wcast-align] sf_iph.c:161:16: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'IP6Frag *' (aka 'struct _IP6Frag *') increases required alignment from 1 to 4 [-Wcast-align] spo_alert_fwsam.c:477:25: warning: cast from 'char *' to 'unsigned long *' increases required alignment from 1 to 4 [-Wcast-align] decode.c:336:13: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'EtherHdr *' (aka 'struct _EtherHdr *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:473:16: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'WifiHdr *' (aka 'struct _WifiHdr *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:634:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:549:33: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'EthLlcOther *' (aka 'struct _EthLlcOther *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:634:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:715:18: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:634:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:634:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:825:13: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'EtherHdr *' (aka 'struct _EtherHdr *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:655:49: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:655:49: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:911:13: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'VlanTagHdr *' (aka 'struct _VlanTagHdr *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:969:29: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'EthLlcOther *' (aka 'struct _EthLlcOther *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:655:49: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:655:49: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:659:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:659:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:659:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:1305:17: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'Trh_llc *' (aka 'struct _Trh_llc *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:1325:20: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'Trh_mr *' (aka 'struct _Trh_mr *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:1339:21: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'Trh_llc *' (aka 'struct _Trh_llc *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:659:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:1345:21: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'Trh_llc *' (aka 'struct _Trh_llc *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:1473:24: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'Fddi_llc_iparp *' (aka 'struct _Fddi_llc_iparp *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:670:49: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:670:49: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:670:49: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:670:49: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:670:49: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:670:49: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:677:53: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:677:53: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:677:53: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:1608:15: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'SLLHdr *' (aka 'struct _SLLHdr *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:677:53: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:677:53: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:1712:15: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'Pflog1Hdr *' (aka 'struct _Pflog1_hdr *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:1791:19: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'Pflog2Hdr *' (aka 'struct _Pflog2_hdr *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:1798:19: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'Pflog3Hdr *' (aka 'struct _Pflog3_hdr *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:1918:27: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'PPPoEHdr *' (aka 'struct _PPPoEHdr *') increases required alignment from 1 to 2 [-Wcast-align] log.c:1725:36: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:677:53: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:2213:28: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'uint16_t *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:2213:28: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'uint16_t *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log.c:1725:36: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:684:47: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:2213:28: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'uint16_t *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log.c:1725:36: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:684:47: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:2213:28: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'uint16_t *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log.c:1725:36: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:684:47: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:2240:14: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'IPHdr *' (aka 'struct _IPHdr *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:2421:24: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'IP4Hdr *' (aka 'struct _IPv4Hdr *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:2422:27: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'IP6RawHdr *' (aka 'struct _IP6RawHdr *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1747:42: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:684:47: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:2549:14: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'uint16_t *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:2549:14: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'uint16_t *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log.c:1747:42: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:684:47: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:2549:14: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'uint16_t *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log.c:1747:42: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:684:47: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:2549:14: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'uint16_t *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log.c:1747:42: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:688:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:2621:13: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'struct enc_header *' increases required alignment from 1 to 4 [-Wcast-align] decode.c:2687:22: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'IPHdr *' (aka 'struct _IPHdr *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:2712:29: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'IPHdr *' (aka 'struct _IPHdr *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1751:36: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:688:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:3030:15: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'TCPHdr *' (aka 'struct _TCPHdr *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1751:36: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:688:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1751:36: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:688:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1751:36: warning: cast from 'u_char *' (aka 'unsigned char *') to 'u_short *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log_text.c:688:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1762:42: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:688:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1762:42: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:695:46: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1762:42: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:695:46: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1762:42: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:695:46: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1762:42: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:695:46: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1762:42: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:695:46: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1769:46: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:695:46: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1769:46: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:702:50: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1769:46: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:702:50: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1769:46: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:702:50: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1769:46: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:702:50: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1769:46: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:702:50: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1776:40: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1776:40: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:702:50: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1776:40: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:709:51: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1776:40: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:709:51: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1776:40: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:709:51: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1776:40: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:709:51: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1780:36: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:709:51: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1780:36: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log_text.c:709:51: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1780:36: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1780:36: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:3211:15: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'UDPHdr *' (aka 'struct _UDPHdr *') increases required alignment from 1 to 2 [-Wcast-align] log.c:1780:36: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1780:36: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1787:39: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:3397:16: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'ICMPHdr *' (aka 'struct _ICMPHdr *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:3554:19: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'IPHdr *' (aka 'struct _IPHdr *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:3645:28: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'TCPHdr *' (aka 'struct _TCPHdr *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:3654:28: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'UDPHdr *' (aka 'struct _UDPHdr *') increases required alignment from 1 to 2 [-Wcast-align] log.c:1787:39: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:3663:29: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'ICMPHdr *' (aka 'struct _ICMPHdr *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1787:39: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1787:39: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1787:39: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:3690:13: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'EtherARP *' (aka 'struct _EtherARP *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:3718:15: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'EtherEapol *' (aka 'struct _EtherEapol *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:3776:15: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'EAPHdr *' (aka 'struct _EAPHdr *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:3814:20: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'IP6Hdr *' (aka 'struct _IPv6Hdr *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1787:39: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:3916:28: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'TCPHdr *' (aka 'struct _TCPHdr *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:3925:28: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'UDPHdr *' (aka 'struct _UDPHdr *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:3934:29: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'ICMPHdr *' (aka 'struct _ICMPHdr *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1794:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:3955:16: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'ICMPHdr *' (aka 'struct _ICMPHdr *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:3993:29: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'ICMP6Hdr *' (aka 'struct _ICMP6 *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:4022:29: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'ICMP6Hdr *' (aka 'struct _ICMP6 *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:4135:44: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'IP6Frag *' (aka 'struct _IP6Frag *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1794:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:4382:11: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'IP6RawHdr *' (aka 'struct _IP6RawHdr *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:4427:26: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'IP6RawHdr *' (aka 'struct _IP6RawHdr *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:4449:29: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'IPHdr *' (aka 'struct _IPHdr *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:4631:15: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'GREHdr *' (aka 'struct _GREHdr *') increases required alignment from 1 to 2 [-Wcast-align] log.c:1794:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:4672:46: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'uint16_t *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] decode.c:4672:46: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'uint16_t *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log.c:1794:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:4672:46: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'uint16_t *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log.c:1794:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:4672:46: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'uint16_t *' (aka 'unsigned short *') increases required alignment from 1 to 2 [-Wcast-align] log.c:1794:43: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] decode.c:4835:13: warning: cast from 'const uint8_t *' (aka 'const unsigned char *') to 'EtherHdr *' (aka 'struct _EtherHdr *') increases required alignment from 1 to 2 [-Wcast-align] log.c:1801:44: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1801:44: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1801:44: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1801:44: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1801:44: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align] log.c:1801:44: warning: cast from 'u_char *' (aka 'unsigned char *') to 'uint32_t *' (aka 'unsigned int *') increases required alignment from 1 to 4 [-Wcast-align]
-
I used both
-O0
and-g
flags. It seems to resolve the issues. I saw the data and event tables got loaded by Barnyard2:MariaDB [snort_db_wan]> select * from event; +-----+-----+-----------+---------------------+ | sid | cid | signature | timestamp | +-----+-----+-----------+---------------------+ | 1 | 4 | 323 | 2019-08-07 18:26:33 | | 1 | 5 | 323 | 2019-08-07 18:28:18 | | 1 | 6 | 323 | 2019-08-07 18:28:21 | | 1 | 7 | 323 | 2019-08-07 18:28:23 | | 1 | 8 | 323 | 2019-08-08 17:23:50 | | 1 | 9 | 323 | 2019-08-08 17:23:55 | | 1 | 10 | 323 | 2019-08-08 17:23:56 | | 1 | 11 | 323 | 2019-08-08 17:42:54 | +-----+-----+-----------+---------------------+
Your suggestion did resolve bus error. I will keep it running over the weekend. If it does fix the issues, I will send a PR.
Thanks!
-
@rickyzhang :
Using the-O0
flag should be sufficient. The-g
option will add debug tables if I recall, so that will bloat the resulting binary a bit. Not a huge deal either way.I figured Barnyard2 was going to be similar to Snort. There are just tons of places where various casts are performed, and if you do something like change structure alignment with the
__packed
attribute, then you can't be sure you have not broken something else in another function that expects the structure to have a particular alignment.All the hassle is a result of what I was sort of ranting about much earlier in this thread. Intel's decision to just always do auto-fixup of unaligned access made programmers (especially C programmers) lazy and inattentive to alignment issues. What I wish now is that all of the other CPU designers would just do the same thing and make auto-fixup happen. There's just too much sloppy code out there now. I get the philosophical argument that unaligned access is a bad idea and that it is sometimes highly inefficient, but fixing all the popular software out there that has sloppy casting is a huge undertaking that nobody wants to tackle. So why not just be practical and always do the auto-fixup?
-
I totally agree with your last statement.
We always face a trade-off between productivity and performance. Now we write as less as possible in low level assembly language because high level abstract language can improve productivity and portability at the cost of performance loss.
Optimization is great in theory but premature one is not in practice. Since ARMv6, ARM cpu supports unaligned memory access in their hardware like intel CPU does.
It is complete clang compiler's fault that they thought they are so smart to generate optimized target code but planting a random timing bomb, instead. In the end, what is more important: optimized executable but randomly fail or reliable executable but slow? The answer is obvious.
-
@rickyzhang said in Barnyard2 and MariaDB:
...Since ARMv6, ARM cpu supports unaligned memory access in their hardware like intel CPU does.
That is not 100% correct. ARMv7 and later support unaligned access for most opcodes, but not all of them. You still have the LDM and STM opcodes (and I think there are two others as well) that never support unaligned access (meaning the CPU microcode engine will not automatically adjust those opcodes for unaligned access).
The SG-3100 actually uses an armv7 processor and it is the clang/llvm compiler's choice of using the LDM and STM opcodes when optimizing that causes the crash. If the armv7 CPU did an auto-fixup for every opcode, then it would not matter which opcodes the compiler chose to use when optimizing.
-
ARM doc already specified that multiple load/store LDM and STM doesn't support unaligned memory access. So why compiler still optimize this operation? This is a typical premature optimization.
The
__packed
qualifier in pointer casting suppose inform compiler don't use optimized load/store. Or put it in this way: use alignment byte = 1 in__packed
qualifier. In this case, the compiler still can use optimized load/store in other suitable use case rather than we completely turn off optimization.I checked the proposed solution to do a proper pointer casting in C to avoid unaligned memory access: copy the source buffer by
memcpy
function. Thememcpy
function always allocate the heap at the maximum multiplier aligned address. So the casting always works. But it is an overhead. You have to do clean up otherwise memory leaks. I don't think it is a simple patch any more.I think it is not ARM but clang should fix their way to generate target machine code.
-
@rickyzhang said in Barnyard2 and MariaDB:
ARM doc already specified that multiple load/store LDM and STM doesn't support unaligned memory access. So why compiler still optimize this operation? This is a typical premature optimization.
The
__packed
qualifier in pointer casting suppose inform compiler don't use optimized load/store. Or put it in this way: use alignment byte = 1 in__packed
qualifier. In this case, the compiler still can use optimized load/store in other suitable use case rather than we completely turn off optimization.I checked the proposed solution to do a proper pointer casting in C to avoid unaligned memory access: copy the source buffer by
memcpy
function. Thememcpy
function always allocate the heap at the maximum multiplier aligned address. So the casting always works. But it is an overhead. You have to do clean up otherwise memory leaks. I don't think it is a simple patch any more.I think it is not ARM but clang should fix their way to generate target machine code.
I agree with you that clang/llvm could do better, but I bet its developers will throw down the philosophical argument that the bad C code should be fixed instead of their compiler generating workarounds for the bad coding, and that what the compiler is doing is "ethically" correct when optimizing as the end result is faster more efficient machine code (of course if that machine code crashes, then just how well off are you?). You will experience zealots on both sides of the argument, unfortunately.
I'm not trying to argue with you. Just pointing out you should probably throw in the towel like I have for Snort and Suricata and just compile your Barnyard2 code on ARM hardware with optimizations disabled.
-
Yes, I'm done now. It is quite a learning process. I really appreciate your advice and guidance.
I disabled optimization on Friday. Barnyard2 ran OK. Crash free for two days. Maria DB are loaded with alerts.
I'm ready to send the PR. See my branch: https://github.com/rickyzhang82/FreeBSD-ports/commits/patch-barnyard2-v2-4-4-3
I patched and tested it on FreeBSD-ports repo v2.4.4_3 release tag with
make.conf
from pfsense repo v2.4.4_3 tag.I saw devel branch in FreeBSD-ports repo and folks send their PR to devel branch. But I don't see devel branch in pfsense.
Please correct me if I'm wrong:
- I should send my PR to FreeBSD-ports on devel branch.
- Use make.conf from pfsense in branch RELENG_2_4_4.
-
@rickyzhang :
I submit my changes against devel in FreeBSD-ports and against master in pfSense. I suspect there are no FreeBSD 11.2 vs FreeBSD 12.0 dependency issues with Barnyard2, so I would submit against devel and master as I mentioned. The pfSense team will then cherry pick and port the changes over to 2.4.4_3 for you. Their preference is to take FreeBSD-ports changes into devel, let them stew in the pfSense-DEVEL branch a bit, and then port them over to RELEASE.What are you doing on the Barnyard2 configuration side (I'm talking about the
barnyard2.conf
file) to use the Maria DB? Is it just a drop-in for MySQL with no changes in the Barnyard2 configuration? If not, then you should also create the necessary Snort and Suricata GUI package patches to make the new DB work with them. -
No Snort configuration GUI change are required. I used their pfsense web UI to configure Barnyard2 . (Although I'd like their UI can configure MariaDB port as well, I lived with 3306 default port)
The package didn't ship with data base initialization script. You have to set up Maria DB and run DB creation manually.