core: crypto: fix AES-GCM in-place decryption order

Fix AES-GCM in-place decryption to ensure GHASH always uses the original
ciphertext. Previously, plaintext could overwrite ciphertext before GHASH

core: crypto: fix AES-GCM in-place decryption order

Fix AES-GCM in-place decryption to ensure GHASH always uses the original
ciphertext. Previously, plaintext could overwrite ciphertext before GHASH,
causing authentication failures. Now GHASH is processed before decryption,
so in-place and non in-place decryption both work correctly without extra
buffering or conditional checks.

Tested with both in-place and non in-place decryption; all cases now
produce correct authentication tags.

Fixes: 1fca7e269b13 ("core: crypto: add new AES-GCM implementation")
Signed-off-by: Rayan Hu <rayan.hu@mediatek.com>
Reviewed-by: Menson Chen <menson.chen@mediatek.com>
Reviewed-by: ChingMing Chen <chingming.chen@mediatek.com>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: crypto: fix internal AES-GCM counter implementation

We have several AES-GCM implementations in crypto libraries and
internal. The internal implementation comes in two flavours, with Arm
crypto

core: crypto: fix internal AES-GCM counter implementation

We have several AES-GCM implementations in crypto libraries and
internal. The internal implementation comes in two flavours, with Arm
crypto extensions (CFG_CRYPTO_WITH_CE=y) and a pure software
implementation.

Each block to be encrypted is xored with an encrypted counter block of
equal size (16 bytes). For each block the counter is increased.

Prior to this patch the entire counter block was increased as a 128-bit
integer, but that's not how AES-GCM is defined. In AES-GCM only the
least significant 32 bits of the counter block are increased, leaving
the rest untouched. The difference is only noticeable when the 32 bits
has reached 0xffffffff and wraps to 0x00000000 on next increment. With a
128-bit integer this would propagate into other parts of the block.

Fix this by only incrementing the last 32-bit word in the counter block,
both in the pure software implementation and when using Arm crypto
extensions.

Link: https://github.com/OP-TEE/optee_os/issues/6659
Fixes: 1fca7e269b13 ("core: crypto: add new AES-GCM implementation")
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Etienne Carriere <etienne.carriere@foss.st.com>

show more ...

core: drop __weak from internal_aes_gcm_update_payload_blocks()

Removes the __weak attribute from internal_aes_gcm_update_payload_blocks()
now that both AArch32 and AArch64 have an optimized replace

core: drop __weak from internal_aes_gcm_update_payload_blocks()

Removes the __weak attribute from internal_aes_gcm_update_payload_blocks()
now that both AArch32 and AArch64 have an optimized replacement.

The previous __weak internal_aes_gcm_update_payload_blocks() is now
moved into core/crypto/aes-gcm-sw.c with its helper functions.

Acked-by: Etienne Carriere <etienne.carriere@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: crypto: remove internal_aes_gcm_expand_enc_key()

Removes internal_aes_gcm_expand_enc_key() which is replaced by
crypto_aes_expand_enc_key().

Reviewed-by: Etienne Carriere <etienne.carriere@li

core: crypto: remove internal_aes_gcm_expand_enc_key()

Removes internal_aes_gcm_expand_enc_key() which is replaced by
crypto_aes_expand_enc_key().

Reviewed-by: Etienne Carriere <etienne.carriere@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: update AArch64 GHASH acceleration routines

Update AArch64 GHASH acceleration routines for improved performance.

The core parts of assembly and wrapper updates are written by
Ard Biesheuvel <a

core: update AArch64 GHASH acceleration routines

Update AArch64 GHASH acceleration routines for improved performance.

The core parts of assembly and wrapper updates are written by
Ard Biesheuvel <ard.biesheuvel@linaro.org>, see [1].

Link: [1] https://github.com/torvalds/linux/commit/22240df7ac6d76a271197571a7be45addef2ba15
Acked-by: Jerome Forissier <jerome@forissier.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: crypto: add internal_aes_gcm_{en,de}crypt_block()

Adds internal_aes_gcm_encrypt_block() and
internal_aes_gcm_decrypt_block() to encrypt or decrypt a well aligned
AES-GCM payload block.

Review

core: crypto: add internal_aes_gcm_{en,de}crypt_block()

Adds internal_aes_gcm_encrypt_block() and
internal_aes_gcm_decrypt_block() to encrypt or decrypt a well aligned
AES-GCM payload block.

Reviewed-by: Etienne Carriere <etienne.carriere@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: crypto: remove internal_aes_gcm_encrypt_block()

Replaces calls to internal_aes_gcm_encrypt_block() with calls to
crypto_aes_enc_block(). Removes internal_aes_gcm_encrypt_block().

Reviewed-by:

core: crypto: remove internal_aes_gcm_encrypt_block()

Replaces calls to internal_aes_gcm_encrypt_block() with calls to
crypto_aes_enc_block(). Removes internal_aes_gcm_encrypt_block().

Reviewed-by: Etienne Carriere <etienne.carriere@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: crypto: unaligned aes-gcm acceleration

The Arm CE code supports working with unaligned data. In order to make
full use of that is the generic __weak function
internal_aes_gcm_update_payload_bl

core: crypto: unaligned aes-gcm acceleration

The Arm CE code supports working with unaligned data. In order to make
full use of that is the generic __weak function
internal_aes_gcm_update_payload_block_aligned() replaced with
internal_aes_gcm_update_payload_blocks(). The latter now supports
working with unaligned buffers.

Acked-by: Jerome Forissier <jerome@forissier.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: crypto: refactor aes-gcm implementation

Adds struct internal_ghash_key to represent the ghash key instead of
some lose fields inside struct internal_aes_gcm_state.

Software of CE configuratio

core: crypto: refactor aes-gcm implementation

Adds struct internal_ghash_key to represent the ghash key instead of
some lose fields inside struct internal_aes_gcm_state.

Software of CE configuration is done explicitly in
core/crypto/aes-gcm-sw.c, dropping the __weak attribute for all
functions but internal_aes_gcm_update_payload_block_aligned() which
is only overridden with CFG_CRYPTO_WITH_CE=y in AArch64.

Content of aes-gcm-private.h is moved into internal_aes-gcm.h.

internal_aes_gcm_gfmul() is made available for generic GF
multiplication.

The CE versions of internal_aes_gcm_expand_enc_key() and
internal_aes_gcm_encrypt_block() are now only wrappers around
crypto_accel_aes_expand_keys() and crypto_accel_aes_ecb_enc().

Acked-by: Jerome Forissier <jerome@forissier.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: crypto: introduce struct crypto_authenc_ops

Uses struct crypto_authenc_ops pointer in crypto context for authenc
ciphers (AES-GCM and AES-CCM) as a glue layer instead of a switch(algo)
in each

core: crypto: introduce struct crypto_authenc_ops

Uses struct crypto_authenc_ops pointer in crypto context for authenc
ciphers (AES-GCM and AES-CCM) as a glue layer instead of a switch(algo)
in each cryto_authenc_*() function.

Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

libutils: remove buf_compare_ct()

Now that we have consttime_memcmp(), buf_compare_ct() is redundant.
Every time buf_compare_ct() is used, consttime_memcmp() may be used
instead.

This commit remove

libutils: remove buf_compare_ct()

Now that we have consttime_memcmp(), buf_compare_ct() is redundant.
Every time buf_compare_ct() is used, consttime_memcmp() may be used
instead.

This commit removes buf_compare_ct(). A compatibility wrapper is kept
in <string_ext.h> to avoid knowingly breaking the build of any TA that
may use it.

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: crypto.h manages authenc context memory

To ease integration with other crypto libraries change the authenc context
interface in crypto.h to manage the memory used for the authenc context.

Rev

core: crypto.h manages authenc context memory

To ease integration with other crypto libraries change the authenc context
interface in crypto.h to manage the memory used for the authenc context.

Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

Reformat copyright/license header in files with an SPDX ID

Some files were committed with an SPDX license identifier before the
rules were defined [1]. Reformat them accordingly.

[1] documentation/

Reformat copyright/license header in files with an SPDX ID

Some files were committed with an SPDX license identifier before the
rules were defined [1]. Reformat them accordingly.

[1] documentation/copyright_and_license_headers.rst

Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Acked-by: Joakim Bech <joakim.bech@linaro.org>

show more ...

core: crypto: AES-GCM: internal_aes_gcm_{enc,dec}()

Adds internal_aes_gcm_enc() and internal_aes_gcm_dec() for
encrypting/decrypting a complete message with an external expanded
key.

Acked-by: Jero

core: crypto: AES-GCM: internal_aes_gcm_{enc,dec}()

Adds internal_aes_gcm_enc() and internal_aes_gcm_dec() for
encrypting/decrypting a complete message with an external expanded
key.

Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: crypto: AES-GCM: separate encryption key

Separates the AES (CTR) encryption key from the rest of the context
to allow more efficient key handling.

Acked-by: Jerome Forissier <jerome.forissier

core: crypto: AES-GCM: separate encryption key

Separates the AES (CTR) encryption key from the rest of the context
to allow more efficient key handling.

Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: crypto: AES-GCM: remove tomcrypt.h dependency

Removes tomcrypt.h dependency by replacing the "symmetric_key skey"
field in struct internal_aes_gcm_ctx with a raw key. Replaces calls to
the LTC

core: crypto: AES-GCM: remove tomcrypt.h dependency

Removes tomcrypt.h dependency by replacing the "symmetric_key skey"
field in struct internal_aes_gcm_ctx with a raw key. Replaces calls to
the LTC functions aes_setup() and aes_ecb_encrypt() with calls to
crypto_aes_expand_enc_key() and crypto_aes_enc_block() respectively.

Reviewed-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: AES-GCM: import table based GF-mult

Imports table based GF multiplication from mbed TLS.

Sets CFG_AES_GCM_TABLE_BASED to default y unless CFG_CRYPTO_WITH_CE is
y, then CFG_AES_GCM_TABLE_BASED

core: AES-GCM: import table based GF-mult

Imports table based GF multiplication from mbed TLS.

Sets CFG_AES_GCM_TABLE_BASED to default y unless CFG_CRYPTO_WITH_CE is
y, then CFG_AES_GCM_TABLE_BASED forced n.

With tables performance is on HiKey960 (CFG_CRYPTO_WITH_CE=n):
xtest --aes-perf -m GCM
(CFG_AES_GCM_TABLE_BASED=n)
min=69.27us max=86.458us mean=70.5695us stddev=0.955826us (cv 1.35445%) (13.8383MiB/s)
(CFG_AES_GCM_TABLE_BASED=y)
min=41.666us max=53.646us mean=42.138us stddev=0.621345us (cv 1.47455%) (23.1753MiB/s)

Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...

core: crypto: add new AES-GCM implementation

Adds a new AES-GCM implementation optimized for hardware acceleration.

This implementation is enabled by default, to use the implementation in
libTomCry

core: crypto: add new AES-GCM implementation

Adds a new AES-GCM implementation optimized for hardware acceleration.

This implementation is enabled by default, to use the implementation in
libTomCrypt instead set CFG_CRYPTO_AES_GCM_FROM_CRYPTOLIB=y.

Tested-by: Jerome Forissier <jerome.forissier@linaro.org> (HiKey960)
Acked-by: Jerome Forissier <jerome.forissier@linaro.org>
Signed-off-by: Jens Wiklander <jens.wiklander@linaro.org>

show more ...