Re: [COMMITTERS] pgsql: Use Intel SSE 4.2 CRC instructions where available.

Use Intel SSE 4.2 CRC instructions where available.

Modern x86 and x86-64 processors with SSE 4.2 support have special
instructions, crc32b and crc32q, for calculating CRC-32C. They greatly
speed up CRC calculation.

Whether the instructions can be used or not depends on the compiler and the
target architecture. If generation of SSE 4.2 instructions is allowed for
the target (-msse4.2 flag on gcc and clang), use them. If they are not
allowed by default, but the compiler supports the -msse4.2 flag to enable
them, compile just the CRC-32C function with -msse4.2 flag, and check at
runtime whether the processor we're running on supports it. If it doesn't,
fall back to the slicing-by-8 algorithm. (With the common defaults on
current operating systems, the runtime-check variant is what you get in
practice.)

Abhijit Menon-Sen, heavily modified by me, reviewed by Andres Freund.

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/3dc2d62d0486325bf263655c2d9a96aee0b02abe

Modified Files
--------------
config/c-compiler.m4 | 27 ++++++
configure | 212 +++++++++++++++++++++++++++++++++++++++++
configure.in | 78 +++++++++++++++
src/Makefile.global.in | 4 +
src/include/pg_config.h.in | 15 +++
src/include/pg_config.h.win32 | 23 ++++-
src/include/port/pg_crc32c.h | 44 +++++++++
src/port/Makefile | 8 +-
src/port/pg_crc32c_choose.c | 63 ++++++++++++
src/port/pg_crc32c_sse42.c | 52 ++++++++++
src/tools/msvc/Mkvcbuild.pm | 13 ++-
11 files changed, 534 insertions(+), 5 deletions(-)

On 14 April 2015 at 10:09, Heikki Linnakangas <heikki(dot)linnakangas(at)iki(dot)fi> wrote:

> Abhijit Menon-Sen, heavily modified by me, reviewed by Andres Freund.

Did the heavy modifications have any affect on the patch behaviour, or
was this just related to where you would like to put the code?

I know you like to rewrite things your way, just wanted to check what
happened. Thanks.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, RemoteDBA, Training & Services

On 04/14/2015 06:28 PM, Simon Riggs wrote:
> On 14 April 2015 at 10:09, Heikki Linnakangas <heikki(dot)linnakangas(at)iki(dot)fi> wrote:
>
>> Abhijit Menon-Sen, heavily modified by me, reviewed by Andres Freund.
>
> Did the heavy modifications have any affect on the patch behaviour, or
> was this just related to where you would like to put the code?

Didn't affect behaviour.

Hmm, the buildfarm animals using Intel C compiler didn't like this
patch. The problem seems to be that unlike on gcc and clang, icc always
has the SSE 4.2 intrinsics (_mm_crc32_u64, _mm_crc32_u8 etc.), even when
the target CPU architecture is not SSE 4.2. On gcc/clang, those
intrinsics are not defined unless you build with -msse4.2.

I'll try to find a fix. I think we could use the __SSE4_2__ define to
check whether SSE4.2 is targeted. Or we can punt and always build the
version with the runtime check, unless overridden manually at configure
command line.

- Heikki

Heikki Linnakangas <heikki(dot)linnakangas(at)iki(dot)fi> writes:
> Whether the instructions can be used or not depends on the compiler and the
> target architecture. If generation of SSE 4.2 instructions is allowed for
> the target (-msse4.2 flag on gcc and clang), use them. If they are not
> allowed by default, but the compiler supports the -msse4.2 flag to enable
> them, compile just the CRC-32C function with -msse4.2 flag, and check at
> runtime whether the processor we're running on supports it. If it doesn't,
> fall back to the slicing-by-8 algorithm. (With the common defaults on
> current operating systems, the runtime-check variant is what you get in
> practice.)

The buildfarm says this doesn't work terribly well, at least not with icc.

regards, tom lane

On 14 April 2015 at 11:34, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:

> Or we can punt and always build the version with
> the runtime check, unless overridden manually at configure command line.

That seems safer as the default, which is what the original patch did.

We can optimise that for known good builds later if desired.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, RemoteDBA, Training & Services

On 04/14/2015 07:01 PM, Simon Riggs wrote:
> On 14 April 2015 at 11:34, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:
>
>> Or we can punt and always build the version with
>> the runtime check, unless overridden manually at configure command line.
>
> That seems safer as the default, which is what the original patch did.
>
> We can optimise that for known good builds later if desired.

I committed the __SSE_4_2__ test for now, so we still build the SSE-only
version if the target allows that. We'll see how that works..

- Heikki