1*4882a593SmuzhiyunThere are expectations of users of the fetcher code. This file attempts to document 2*4882a593Smuzhiyunsome of the constraints that are present. Some are obvious, some are less so. It is 3*4882a593Smuzhiyundocumented in the context of how OE uses it but the API calls are generic. 4*4882a593Smuzhiyun 5*4882a593Smuzhiyuna) network access for sources is only expected to happen in the do_fetch task. 6*4882a593Smuzhiyun This is not enforced or tested but is required so that we can: 7*4882a593Smuzhiyun 8*4882a593Smuzhiyun i) audit the sources used (i.e. for license/manifest reasons) 9*4882a593Smuzhiyun ii) support offline builds with a suitable cache 10*4882a593Smuzhiyun iii) allow work to continue even with downtime upstream 11*4882a593Smuzhiyun iv) allow for changes upstream in incompatible ways 12*4882a593Smuzhiyun v) allow rebuilding of the software in X years time 13*4882a593Smuzhiyun 14*4882a593Smuzhiyunb) network access is not expected in do_unpack task. 15*4882a593Smuzhiyun 16*4882a593Smuzhiyunc) you can take DL_DIR and use it as a mirror for offline builds. 17*4882a593Smuzhiyun 18*4882a593Smuzhiyund) access to the network is only made when explicitly configured in recipes 19*4882a593Smuzhiyun (e.g. use of AUTOREV, or use of git tags which change revision). 20*4882a593Smuzhiyun 21*4882a593Smuzhiyune) fetcher output is deterministic (i.e. if you fetch configuration XXX now it 22*4882a593Smuzhiyun will match in future exactly in a clean build with a new DL_DIR). 23*4882a593Smuzhiyun One specific pain point example are git tags. They can be replaced and change 24*4882a593Smuzhiyun so the git fetcher has to resolve them with the network. We use git revisions 25*4882a593Smuzhiyun where possible to avoid this and ensure determinism. 26*4882a593Smuzhiyun 27*4882a593Smuzhiyunf) network access is expected to work with the standard linux proxy variables 28*4882a593Smuzhiyun so that access behind firewalls works (the fetcher sets these in the 29*4882a593Smuzhiyun environment but only in the do_fetch tasks). 30*4882a593Smuzhiyun 31*4882a593Smuzhiyung) access during parsing has to be minimal, a "git ls-remote" for an AUTOREV 32*4882a593Smuzhiyun git recipe might be ok but you can't expect to checkout a git tree. 33*4882a593Smuzhiyun 34*4882a593Smuzhiyunh) we need to provide revision information during parsing such that a version 35*4882a593Smuzhiyun for the recipe can be constructed. 36*4882a593Smuzhiyun 37*4882a593Smuzhiyuni) versions are expected to be able to increase in a way which sorts allowing 38*4882a593Smuzhiyun package feeds to operate (see PR server required for git revisions to sort). 39*4882a593Smuzhiyun 40*4882a593Smuzhiyunj) API to query for possible version upgrades of a url is highly desireable to 41*4882a593Smuzhiyun allow our automated upgrage code to function (it is implied this does always 42*4882a593Smuzhiyun have network access). 43*4882a593Smuzhiyun 44*4882a593Smuzhiyunk) Where fixes or changes to behaviour in the fetcher are made, we ask that 45*4882a593Smuzhiyun test cases are added (run with "bitbake-selftest bb.tests.fetch"). We do 46*4882a593Smuzhiyun have fairly extensive test coverage of the fetcher as it is the only way 47*4882a593Smuzhiyun to track all of its corner cases, it still doesn't give entire coverage 48*4882a593Smuzhiyun though sadly. 49*4882a593Smuzhiyun 50*4882a593Smuzhiyunl) If using tools during parse time, they will have to be in ASSUME_PROVIDED 51*4882a593Smuzhiyun in OE's context as we can't build git-native, then parse a recipe and use 52*4882a593Smuzhiyun git ls-remote. 53*4882a593Smuzhiyun 54*4882a593SmuzhiyunNot all fetchers support all features, autorev is optional and doesn't make 55*4882a593Smuzhiyunsense for some. Upgrade detection means different things in different contexts 56*4882a593Smuzhiyuntoo. 57*4882a593Smuzhiyun 58