Merge changes from topic "fix-windows-builds" into integration

* changes:
fix(build): shell-escape toolchain wrapper commands
fix(build): fix incorrect parentheses expansion in `shell-map`

fix(build): fix incorrect parentheses expansion in `shell-map`

The previous commit which attempted to fix the expansions in `shell-map`
doubled up on the dollar signs for all of the escape substitut

fix(build): fix incorrect parentheses expansion in `shell-map`

The previous commit which attempted to fix the expansions in `shell-map`
doubled up on the dollar signs for all of the escape substitutions; this
"fixed" the Windows build.

However, the expansion of the parentheses is actually still wrong.
Consider the following example:

print = $(warning $(1))

$(call shell-map,print,'$$' '$(lparen)' '$(rparen)')

... which prints:

make_helpers/utilities.mk:620: $
make_helpers/utilities.mk:620: ${lparen}
make_helpers/utilities.mk:620: ${rparen}

However, what we expect to see is:

make_helpers/utilities.mk:620: $
make_helpers/utilities.mk:620: (
make_helpers/utilities.mk:620: )

The reason we do these substitutions is because, behind the scenes, the
function generates a small snippet of Make which calls the map function
provided by the user. To do that safely, we need to escape characters
which can cause premature expansion (`$`), and any characters which can
interfere with the syntax of the `call` function (`(` and `)`).

The shell snippet that we *expected* the example above to generate was:

$(call print,$$,1)
$(call print,${lparen},2)
$(call print,${rparen},3)

However, as of the last "fix", what it is actually generating is:

$(call print,$$,1)
$(call print,$${lparen},2)
$(call print,$${rparen},3)

This breaks the Windows build again but that's because this bug was
actually three bugs from the start.

Change-Id: Ibaf044806101334ddf4080df8da18f9aac8667e5
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge "fix(build): fix incorrect expansions in `shell-map`" into integration

fix(build): fix incorrect expansions in `shell-map`

The `shell-map` function needs to pass values it receives from the shell
back into Make for evaluation, which requires certain special characters

fix(build): fix incorrect expansions in `shell-map`

The `shell-map` function needs to pass values it receives from the shell
back into Make for evaluation, which requires certain special characters
to be escaped.

This escaping is currently resulting in those characters *not* being
escaped, and consequently instead generates some funky (unhelpful)
syntax errors half-way through reading the Makefile.

This change corrects the escape substitutions so that they are evaluated
as expected within Make.

Change-Id: I7b47a48342f626efe362e05283ee59520673375f
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge changes from topic "fix-compiler-wrappers" into integration

* changes:
fix(build): fix compiler wrapper detection
feat(build): update `shell-map` to also pass indices

fix(build): fix compiler wrapper detection

A late change in the recently-merged compiler wrapper detection change
stack introduced two issues:

Firstly, the `irange` function - a dependency of the c

fix(build): fix compiler wrapper detection

A late change in the recently-merged compiler wrapper detection change
stack introduced two issues:

Firstly, the `irange` function - a dependency of the compiler wrapper
detection implementation - was taken out of the change stack, which
meant uses of it evaluated to empty.

Secondly, the shell command used by the `shell-slice` function was
replaced with an alternative implementation which incorrectly sliced the
shell words inclusively rather than exclusively.

This change resolves the issues caused by these changes by replacing the
use of the `irange` function with `shell-map`, and by ensuring that the
`shell-slice` function correctly uses an exclusive end value.

Change-Id: Ic6ef007d3a3c5da1152775634fbeb5fc6ccd41d8
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

feat(build): update `shell-map` to also pass indices

This small change updates the `shell-map` function to also provide the
word index to the map function.

Change-Id: I636dcdb9945681208b9b64a7532a3

feat(build): update `shell-map` to also pass indices

This small change updates the `shell-map` function to also provide the
word index to the map function.

Change-Id: I636dcdb9945681208b9b64a7532a31f6130ac500
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge changes from topic "ck/tf-a/shell-utilities" into integration

* changes:
feat(build): add shell program reference utility (`shell-program`)
feat(build): add shell word map utility (`shell-

Merge changes from topic "ck/tf-a/shell-utilities" into integration

* changes:
feat(build): add shell program reference utility (`shell-program`)
feat(build): add shell word map utility (`shell-map`)
feat(build): add shell fragment joining utility (`shell-join`)
feat(build): add shell fragment slicing utility (`shell-slice`)
feat(build): add shell fragment length utility (`shell-words`)
feat(build): add shell word extraction utility (`shell-word`)
feat(build): add shell quoting utility (`shell-quote`)
feat(build): add temporary variable binding utility (`with`)

show more ...

feat(build): add shell program reference utility (`shell-program`)

This change adds the `shell-program` function to the build system, which
accepts a program name, path or shell fragment and ensures

feat(build): add shell program reference utility (`shell-program`)

This change adds the `shell-program` function to the build system, which
accepts a program name, path or shell fragment and ensures it is safely
and correctly escaped for use in shell command.

Change-Id: I31d00e6da0cc2fd464bb794c7efb8d6ff3685c4b
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

feat(build): add shell word map utility (`shell-map`)

This change adds the `shell-map` function to the build system, which
applies a function to every shell word in a shell fragment.

Change-Id: I20

feat(build): add shell word map utility (`shell-map`)

This change adds the `shell-map` function to the build system, which
applies a function to every shell word in a shell fragment.

Change-Id: I2053e9741876cfa25878a7cadfbecb6b1127fc19
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

feat(build): add shell fragment joining utility (`shell-join`)

This change adds the `shell-join` function to the build system, which
parses a shell fragment and joins it shell words with an optional

feat(build): add shell fragment joining utility (`shell-join`)

This change adds the `shell-join` function to the build system, which
parses a shell fragment and joins it shell words with an optional user-
specified delimiter.

Change-Id: I201f728c2101bf67f14a037454a7b17d9b1c34a2
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

feat(build): add shell fragment slicing utility (`shell-slice`)

This change adds the `shell-slice` function to the build system, which
parses a shell fragment and extracts a sequence of shell words,

feat(build): add shell fragment slicing utility (`shell-slice`)

This change adds the `shell-slice` function to the build system, which
parses a shell fragment and extracts a sequence of shell words, returned
as a new shell fragment.

Change-Id: I0df1e451bae6b973711e26c75b435939251aeb51
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

feat(build): add shell fragment length utility (`shell-words`)

This change adds the `shell-words` function to the build system, which
parses a shell fragment and counts the number of shell words in

feat(build): add shell fragment length utility (`shell-words`)

This change adds the `shell-words` function to the build system, which
parses a shell fragment and counts the number of shell words in it.

Change-Id: Ief031efd5e013a74e6e5a09fc8e8382c4d47e591
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

feat(build): add shell word extraction utility (`shell-word`)

This change adds the `shell-word` function to the build system, which
parses a shell fragment and extracts the N-th shell word.

Change-

feat(build): add shell word extraction utility (`shell-word`)

This change adds the `shell-word` function to the build system, which
parses a shell fragment and extracts the N-th shell word.

Change-Id: Ie35d5164b58188eade851bbcc3f700e51c0837df
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

feat(build): add shell quoting utility (`shell-quote`)

This change adds the `shell-quote` function to the build system, which
currently duplicates the existing behaviour of `escape-shell`, but
estab

feat(build): add shell quoting utility (`shell-quote`)

This change adds the `shell-quote` function to the build system, which
currently duplicates the existing behaviour of `escape-shell`, but
establishes a consistent naming style for upcoming shell utilities.

Change-Id: Iae710be4d9b57e4d0f1d4dae9985a0e6af8f7b86
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

feat(build): add temporary variable binding utility (`with`)

This change adds the `with` function to the build system, which provides
lexically-scoped (temporary) variable bindings during the expans

feat(build): add temporary variable binding utility (`with`)

This change adds the `with` function to the build system, which provides
lexically-scoped (temporary) variable bindings during the expansion of a
body of text. This largely mirrors the behaviour of the `let` function
introduced in GNU Make 4.4 while being compatible with earlier versions.

Change-Id: I83df2be34a7b4e48f31cdf1d615a4b4955299740
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge changes from topic "hm/evlog" into integration

* changes:
build(measured-boot)!: move to ext event log lib
feat(build): add utilities for modifying includes

feat(build): add utilities for modifying includes

Add Makefile helpers to manipulate and extract include directories.
Introduce strip-quotes, include-dirs, and include-dirs-realpath
utilities for fl

feat(build): add utilities for modifying includes

Add Makefile helpers to manipulate and extract include directories.
Introduce strip-quotes, include-dirs, and include-dirs-realpath
utilities for flexible processing of -I flags.

Change-Id: Ic8051d299b43e7efeca4cccd8c36115bc8393bcf
Signed-off-by: Harrison Mutai <harrison.mutai@arm.com>
Co-authored-by: Chris Kay <Chris.Kay@arm.com>

show more ...

Merge "build: remove Windows compatibility layer" into integration

build: remove Windows compatibility layer

For a couple of releases now we have officially withdrawn support for
building TF-A on Windows using the native environment, relying instead
on POSIX emulat

build: remove Windows compatibility layer

For a couple of releases now we have officially withdrawn support for
building TF-A on Windows using the native environment, relying instead
on POSIX emulation layers like MSYS2, Mingw64, Cygwin or WSL.

This change removes the remainder of the OS compatibility layer
entirely, and migrates the build system over to explicitly relying on a
POSIX environment.

Change-Id: I8fb60d998162422e958009afd17eab826e3bc39b
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge "build: fix grouped targets on Make <= 4.2" into integration

build: fix grouped targets on Make <= 4.2

Grouped targets are a feature introduced with GNU Make 4.3 which enable
rules with multiple targets to communicate that all of the targets of
that rule are

build: fix grouped targets on Make <= 4.2

Grouped targets are a feature introduced with GNU Make 4.3 which enable
rules with multiple targets to communicate that all of the targets of
that rule are built simultaneously, rather than independently.

For example, without grouped targets the following rule may be executed
twice:

a.txt b.txt:
touch a.txt b.txt

# $ remake -j2 a.txt b.txt
# touch a.txt b.txt
# touch a.txt b.txt

In this example, both `a.txt` and `b.txt` are touched twice, when the
rule should only be executed once. Instead, this rule can use a grouped
target:

a.txt b.txt &:
touch a.txt b.txt

# $ remake -j2 a.txt b.txt
# touch a.txt b.txt
# remake: 'b.txt' is up to date.

In this case, both `a.txt` and `b.txt` are created once only, as Make
now knows that the recipe will create both files.

Note that pattern rules with multiple targets always behave this way.

Previously, we assumed that the grouped target feature was available,
but on systems still packaging Make 4.2, most prominently Ubuntu 20.04,
this is not the case. This change adds a check to ensure that we do not
use grouped targets if they are unavailable.

Change-Id: Ifd9da35421ae11468d7a25d3cbc76f6036921749
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge "build: make Poetry optional" into integration

build: make Poetry optional

The Yocto team has requested that we do not use Poetry from within the
Makefile, as Yocto does not have network access during the build
process.

We want to maintain the

build: make Poetry optional

The Yocto team has requested that we do not use Poetry from within the
Makefile, as Yocto does not have network access during the build
process.

We want to maintain the current behaviour, so this change makes our use
of Poetry contigent on it being available in the environment.

Additionally, explicitly passing an empty toolchain parameter now allows
a tool to be *disabled* (e.g. passing `POETRY=` will prevent the build
system from trying to use Poetry).

Change-Id: Ibf552a3fee1eaadee767a1b948b559700083b401
Signed-off-by: Chris Kay <chris.kay@arm.com>

show more ...

Merge changes from topic "ck/tf-a/verbosity-cleanup" into integration

* changes:
build: unify verbosity handling
build: add facilities for interpreting boolean values
build: add string casing

Merge changes from topic "ck/tf-a/verbosity-cleanup" into integration

* changes:
build: unify verbosity handling
build: add facilities for interpreting boolean values
build: add string casing facilities to utilities

show more ...