Merge changes from topic "fix-windows-builds" into integration

* changes:
fix(build): shell-escape toolchain wrapper commands
fix(build): fix incorrect parentheses expansion in `shell-map`

fix(build): shell-escape toolchain wrapper commands

When we split toolchain tool commands into their wrapper and program
shell fragments, we do so by iterating over each shell word and
forwarding it

fix(build): shell-escape toolchain wrapper commands

When we split toolchain tool commands into their wrapper and program
shell fragments, we do so by iterating over each shell word and
forwarding it to `toolchain-guess-tool`.

However, we receive each word in its raw form, but we pass it to a
function which expects a fragment. If the raw word contains characters
which are syntactically-meaningful to the shell, then the command can
misbehave, and the build system can unwittingly do some funky things.

This small change just ensures that we re-quote the shell word we
receive before forwarding it on.

Change-Id: I325e1c135fee97d749da927c08e181775b14d146
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge changes from topic "fix-compiler-wrappers" into integration

* changes:
fix(build): fix compiler wrapper detection
feat(build): update `shell-map` to also pass indices

fix(build): fix compiler wrapper detection

A late change in the recently-merged compiler wrapper detection change
stack introduced two issues:

Firstly, the `irange` function - a dependency of the c

fix(build): fix compiler wrapper detection

A late change in the recently-merged compiler wrapper detection change
stack introduced two issues:

Firstly, the `irange` function - a dependency of the compiler wrapper
detection implementation - was taken out of the change stack, which
meant uses of it evaluated to empty.

Secondly, the shell command used by the `shell-slice` function was
replaced with an alternative implementation which incorrectly sliced the
shell words inclusively rather than exclusively.

This change resolves the issues caused by these changes by replacing the
use of the `irange` function with `shell-map`, and by ensuring that the
`shell-slice` function correctly uses an exclusive end value.

Change-Id: Ic6ef007d3a3c5da1152775634fbeb5fc6ccd41d8
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge changes from topic "ck/tf-a/cmake-cc-fix" into integration

* changes:
fix(build): forward compiler launcher to CMake
fix(build): force CMake to cross-compile
fix(build): correctly forwar

Merge changes from topic "ck/tf-a/cmake-cc-fix" into integration

* changes:
fix(build): forward compiler launcher to CMake
fix(build): force CMake to cross-compile
fix(build): correctly forward C compiler to CMake
feat(build): add helpers for managing compiler launchers
refactor(build): track toolchain tool origin
refactor(build): unify toolchain derivation

show more ...

feat(build): add helpers for managing compiler launchers

With the introduction of the Event Log library, we now need to forward
some of our toolchain information to CMake. However, CMake always trea

feat(build): add helpers for managing compiler launchers

With the introduction of the Event Log library, we now need to forward
some of our toolchain information to CMake. However, CMake always treats
the first shell word in `CC` as the path to the compiler, which is
problematic for users of compiler launchers like `ccache`.

This change introduces some additional heuristics to the toolchain
interface to detect compiler launchers and isolate their arguments from
the underlying compiler command, which can then be forwarded separately
to CMake.

Change-Id: I783833c523380b989bbac05b57503bc76881f6a4
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

refactor(build): track toolchain tool origin

If a toolchain tool cannot be identified, the warning now includes a
report describing whether the program was given by a user-specified
parameter, the C

refactor(build): track toolchain tool origin

If a toolchain tool cannot be identified, the warning now includes a
report describing whether the program was given by a user-specified
parameter, the C compiler, or a default value.

Change-Id: Ic71ad33d3123f17433a3c176a93310b39bfa5fff
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

refactor(build): unify toolchain derivation

At the moment, the process we go through to derive the value for the
toolchain tool variables is fairly complicated and not particularly
consistent. This

refactor(build): unify toolchain derivation

At the moment, the process we go through to derive the value for the
toolchain tool variables is fairly complicated and not particularly
consistent. This change brings the three mechanisms we support into
alignment in terms of their inputs and outputs via three helpers:

- `toolchain-from-parameter`: load the command from its parameter
- `toolchain-from-cc`: load the command from its C compiler
- `toolchain-from-default`: load the command from its built-in defaults

These functions take a toolchain identifier and a tool class identifer,
and return either a program name/path or a shell command. Sanitisation
is then done inside `toolchain-determine-tool`, which now makes a
stronger guarantee that the toolchain tools are correctly escaped for
the shell.

Change-Id: I3ce73134824e805e8ee47a119c982e7fe4923fae
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge changes Ia8b52237,Id4b7714e into integration

* changes:
fix(build): put the -target definitions in toolchain.mk
fix(build): align the cpu-ops flags with all others

fix(build): put the -target definitions in toolchain.mk

Knowing the target is quite important and must be known early - it is
necessary for flag discovery and should be available much earlier than
c

fix(build): put the -target definitions in toolchain.mk

Knowing the target is quite important and must be known early - it is
necessary for flag discovery and should be available much earlier than
cflags.mk's inclusion. So put it in toolchain.mk with the rest of the
toolchain configuration.

Change-Id: Ia8b522376d27171ad1282d05c162cddc0fca69ab
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge changes Id711e387,I531a2ee1,Ic5b48514,I81f5f663,I6c529c13, ... into integration

* changes:
refactor(romlib): absorb WRAPPER_FLAGS into LDFLAGS
fix(build): simplify the -target options
fe

Merge changes Id711e387,I531a2ee1,Ic5b48514,I81f5f663,I6c529c13, ... into integration

* changes:
refactor(romlib): absorb WRAPPER_FLAGS into LDFLAGS
fix(build): simplify the -target options
feat(build): allow full LTO builds with clang
refactor(build): make sorting of sections generic
feat(build): use clang as a linker
fix(build): correctly detect that an option is missing with ld_option
feat(build): pass cflags to the linker when LTO is enabled

show more ...

feat(build): use clang as a linker

To support LTO, the gcc binary is used as a compiler, assembler, and
linker. Do the same for clang and enable LTO builds with it as a side
effect.

This simplifies

feat(build): use clang as a linker

To support LTO, the gcc binary is used as a compiler, assembler, and
linker. Do the same for clang and enable LTO builds with it as a side
effect.

This simplifies code quite a bit as the gcc/clang different is much
smaller. Support for ld/lld (if overriden with LD) is maintained.

This is a good time to convert tabs to spaces to conform to make's
expectations on syntax.

Change-Id: I6c529c1393f7e9e8046ed537f871fc3ad91d599a
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

fix(build): correctly detect that an option is missing with ld_option

We support building directly with ld and indirectly with gcc. The
`ld_option` macro is oblivious to this and does a check for bo

fix(build): correctly detect that an option is missing with ld_option

We support building directly with ld and indirectly with gcc. The
`ld_option` macro is oblivious to this and does a check for both styles
of invocation. However, the gcc one is incorrect - gcc returns `0` even
when it has printed an error saying that it doesn't recognise the
option. Add a discovery function for each linker we expect and
dynamically dispatch to the correct one.

While we're at it, also add a little bit of code to return the -Wl
prefix for gcc and not for ld.

All of the above is also true for clang and lld, although they don't
suffer from the problem that gcc does.

Change-Id: I4f7bdf40c01f4c5df9c177f5048f5e349bc2b9c9
Co-authored-by: Chris Kay <chris.kay@arm.com>
Signed-off-by: Boyan Karatotev <boyan.karatotev@arm.com>

show more ...

Merge "build: remove Windows compatibility layer" into integration

build: remove Windows compatibility layer

For a couple of releases now we have officially withdrawn support for
building TF-A on Windows using the native environment, relying instead
on POSIX emulat

build: remove Windows compatibility layer

For a couple of releases now we have officially withdrawn support for
building TF-A on Windows using the native environment, relying instead
on POSIX emulation layers like MSYS2, Mingw64, Cygwin or WSL.

This change removes the remainder of the OS compatibility layer
entirely, and migrates the build system over to explicitly relying on a
POSIX environment.

Change-Id: I8fb60d998162422e958009afd17eab826e3bc39b
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge "build: make Poetry optional" into integration

build: make Poetry optional

The Yocto team has requested that we do not use Poetry from within the
Makefile, as Yocto does not have network access during the build
process.

We want to maintain the

build: make Poetry optional

The Yocto team has requested that we do not use Poetry from within the
Makefile, as Yocto does not have network access during the build
process.

We want to maintain the current behaviour, so this change makes our use
of Poetry contigent on it being available in the environment.

Additionally, explicitly passing an empty toolchain parameter now allows
a tool to be *disabled* (e.g. passing `POETRY=` will prevent the build
system from trying to use Poetry).

Change-Id: Ibf552a3fee1eaadee767a1b948b559700083b401
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge changes from topic "dynamic-toolchain" into integration

* changes:
build: allow multiple toolchain defaults
build: determine toolchain tools dynamically

Merge "build: properly namespace `toolchain.mk` variables" into integration

build: allow multiple toolchain defaults

This change enables a fairly commonly-requested use-case, which is to
fall back to the host system's native toolchain when building on AArch64
if the bare-me

build: allow multiple toolchain defaults

This change enables a fairly commonly-requested use-case, which is to
fall back to the host system's native toolchain when building on AArch64
if the bare-metal toolchain is not available.

In this situation, if the `aarch64-none-elf` GCC toolchain cannot be
located, the build system will look for `aarch64-linux-gnu` before
giving up.

Change-Id: I39d2a8837b651b28cf0eafa92f6003a7f66767a0
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

build: determine toolchain tools dynamically

Since the introduction of the toolchain detection framework into the
build system, we have done determination and identification of the
toolchain(s) used

build: determine toolchain tools dynamically

Since the introduction of the toolchain detection framework into the
build system, we have done determination and identification of the
toolchain(s) used for the build at the initialization of the build
system.

This incurs a large cost to the build every time - for every toolchain
that has been requested by the current makefile, we try to identify each
tool in the list of known tool classes, even if that tool doesn't
actually see any use.

For the clean and check-like targets we worked around this by disabling
most of the toolchains if we detect these targets, but this is
inflexible and not very reliable, and it still means that when building
normal targets we are incurring that cost for all tools whether they are
used or not.

This change instead modifies the toolchain detection framework to only
initialize a tool for a given toolchain when it is first used. This does
mean that we can no longer warn about an incorrectly-configured
toolchain at the beginning of build system invocation, but it has the
advantage of substantially reducing build time and the complexity of
*using* the framework (at the cost of an increase in complexity in the
framework itself).

Change-Id: I7f3d06b2eb58c1b26a846791a13b0037f32c8013
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

build: properly namespace `toolchain.mk` variables

This is a simple change to ensure that everything in the `toolchain.mk`
makefile is adequately namespaced to avoid collisions.

Change-Id: I0b45966

build: properly namespace `toolchain.mk` variables

This is a simple change to ensure that everything in the `toolchain.mk`
makefile is adequately namespaced to avoid collisions.

Change-Id: I0b45966527e1a03200bbc6694fd6c7133e349314
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge "build: use `ar` over `gcc-ar`" into integration

build: use `ar` over `gcc-ar`

It has been a sufficiently long time since the last release of binutils
did not automatically enable the LTO plugin. Migrate to `ar` rather than
using the `gcc-ar` buil

build: use `ar` over `gcc-ar`

It has been a sufficiently long time since the last release of binutils
did not automatically enable the LTO plugin. Migrate to `ar` rather than
using the `gcc-ar` build wrapper, which saves us some pain trying to
locate the proper archiver.

Change-Id: I6f8b895d6a470d2b7cd5b98ccb23c54b35d7ad12
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge "build: allow shell commands in `CC` and friends" into integration