Lines Matching refs:cpuset
48 the resources within a task's current cpuset. They form a nested
59 policy, are both filtered through that task's cpuset, filtering out any
60 CPUs or Memory Nodes not in that cpuset. The scheduler will not
67 cpusets and which CPUs and Memory Nodes are assigned to each cpuset,
68 specify and query to which cpuset a task is assigned, and list the
69 task pids assigned to a cpuset.
103 The kernel cpuset patch provides the minimum essential kernel
124 - Each task in the system is attached to a cpuset, via a pointer
127 allowed in that task's cpuset.
129 those Memory Nodes allowed in that task's cpuset.
130 - The root cpuset contains all the systems CPUs and Memory
132 - For any cpuset, one can define child cpusets containing a subset
134 - The hierarchy of cpusets can be mounted at /dev/cpuset, for
136 - A cpuset may be marked exclusive, which ensures that no other
137 cpuset (except direct ancestors and descendants) may contain
139 - You can list all the tasks (by pid) attached to any cpuset.
144 - in init/main.c, to initialize the root cpuset at system boot.
145 - in fork and exit, to attach and detach a task from its cpuset.
147 allowed in that task's cpuset.
149 the CPUs allowed by their cpuset, if possible.
151 Memory Nodes by what's allowed in that task's cpuset.
153 - in vmscan.c, to restrict page recovery to the current cpuset.
158 modifying cpusets is via this cpuset file system.
170 Each cpuset is represented by a directory in the cgroup file system
172 files describing that cpuset:
174 - cpuset.cpus: list of CPUs in that cpuset
175 - cpuset.mems: list of Memory Nodes in that cpuset
176 - cpuset.memory_migrate flag: if set, move pages to cpusets nodes
177 - cpuset.cpu_exclusive flag: is cpu placement exclusive?
178 - cpuset.mem_exclusive flag: is memory placement exclusive?
179 - cpuset.mem_hardwall flag: is memory allocation hardwalled
180 - cpuset.memory_pressure: measure of how much paging pressure in cpuset
181 - cpuset.memory_spread_page flag: if set, spread page cache evenly on allowed nodes
182 - cpuset.memory_spread_slab flag: if set, spread slab cache evenly on allowed nodes
183 - cpuset.sched_load_balance flag: if set, load balance within CPUs on that cpuset
184 - cpuset.sched_relax_domain_level: the searching range when migrating tasks
186 In addition, only the root cpuset has the following file:
188 - cpuset.memory_pressure_enabled flag: compute memory_pressure?
191 command. The properties of a cpuset, such as its flags, allowed
199 children of that task, to a cpuset allows organizing the work load
201 to using the CPUs and Memory Nodes of a particular cpuset. A task
202 may be re-attached to any other cpuset, if allowed by the permissions
203 on the necessary cpuset file system directories.
209 The following rules apply to each cpuset:
218 exclusive cpuset. Also, the use of a Linux virtual file system (vfs)
219 to represent the cpuset hierarchy provides for a familiar permission
222 The cpus and mems files in the root (top_cpuset) cpuset are
228 The cpuset.effective_cpus and cpuset.effective_mems files are
229 normally read-only copies of cpuset.cpus and cpuset.mems files
230 respectively. If the cpuset cgroup filesystem is mounted with the
232 similar to the corresponding files in cpuset v2. In other words, hotplug
233 events will not change cpuset.cpus and cpuset.mems. Those events will
234 only affect cpuset.effective_cpus and cpuset.effective_mems which show
235 the actual cpus and memory nodes that are currently used by this cpuset.
237 cpuset v2 behavior.
243 If a cpuset is cpu or mem exclusive, no other cpuset, other than
247 A cpuset that is cpuset.mem_exclusive *or* cpuset.mem_hardwall is "hardwalled",
253 isolating each job's user allocation in its own cpuset. To do this,
254 construct a large mem_exclusive cpuset to hold all the jobs, and
258 mem_exclusive cpuset.
263 The memory_pressure of a cpuset provides a simple per-cpuset metric
264 of the rate that the tasks in a cpuset are attempting to free up in
265 use memory on the nodes of the cpuset to satisfy additional memory
280 to monitor a cpuset for signs of memory pressure. It's up to the
286 /dev/cpuset/memory_pressure_enabled, the hook in the rebalance
291 Why a per-cpuset, running average:
293 Because this meter is per-cpuset, rather than per-task or mm,
303 Because this meter is per-cpuset rather than per-task or mm,
305 pressure in a cpuset, with a single read, rather than having to
307 set of tasks in the cpuset.
309 A per-cpuset simple digital filter (requires a spinlock and 3 words
310 of data per-cpuset) is kept, and updated by any task attached to that
311 cpuset, if it enters the synchronous (direct) page reclaim code.
313 A per-cpuset file provides an integer number representing the recent
315 the tasks in the cpuset, in units of reclaims attempted per second,
321 There are two boolean flag files per cpuset that control where the
323 kernel data structures. They are called 'cpuset.memory_spread_page' and
324 'cpuset.memory_spread_slab'.
326 If the per-cpuset boolean flag file 'cpuset.memory_spread_page' is set, then
331 If the per-cpuset boolean flag file 'cpuset.memory_spread_slab' is set,
342 except perhaps as modified by the task's NUMA mempolicy or cpuset
356 Both 'cpuset.memory_spread_page' and 'cpuset.memory_spread_slab' are boolean flag
358 for that cpuset. If a "1" is written to that file, then that turns
363 Setting the flag 'cpuset.memory_spread_page' turns on a per-process flag
364 PFA_SPREAD_PAGE for each task that is in that cpuset or subsequently
365 joins that cpuset. The page allocation calls for the page cache
370 Similarly, setting 'cpuset.memory_spread_slab' turns on the flag
384 the several nodes in the jobs cpuset in order to fit. Without this
386 data set, the memory allocation across the nodes in the jobs cpuset
426 When the per-cpuset flag "cpuset.sched_load_balance" is enabled (the default
427 setting), it requests that all the CPUs in that cpusets allowed 'cpuset.cpus'
430 from any CPU in that cpuset to any other.
432 When the per-cpuset flag "cpuset.sched_load_balance" is disabled, then the
433 scheduler will avoid load balancing across the CPUs in that cpuset,
434 --except-- in so far as is necessary because some overlapping cpuset
437 So, for example, if the top cpuset has the flag "cpuset.sched_load_balance"
439 CPUs, and the setting of the "cpuset.sched_load_balance" flag in any other
442 Therefore in the above two situations, the top cpuset flag
443 "cpuset.sched_load_balance" should be disabled, and only some of the smaller,
447 the top cpuset that might use non-trivial amounts of CPU, as such tasks
454 Of course, tasks pinned to a particular CPU can be left in a cpuset
455 that disables "cpuset.sched_load_balance" as those tasks aren't going anywhere
465 overlapping cpusets enables the flag 'cpuset.sched_load_balance', then we
467 a task to a CPU outside its cpuset, but the scheduler load balancing
471 between which cpusets have the flag "cpuset.sched_load_balance" enabled,
472 and the sched domain configuration. If a cpuset enables the flag, it
475 cpuset enables the flag.
477 If two cpusets have partially overlapping 'cpuset.cpus' allowed, and only
481 paragraphs above. In the general case, as in the top cpuset case,
487 CPUs in "cpuset.isolcpus" were excluded from load balancing by the
489 of the value of "cpuset.sched_load_balance" in any cpuset.
494 The per-cpuset flag 'cpuset.sched_load_balance' defaults to enabled (contrary
495 to most cpuset flags.) When enabled for a cpuset, the kernel will
496 ensure that it can load balance across all the CPUs in that cpuset
497 (makes sure that all the CPUs in the cpus_allowed of that cpuset are
500 If two overlapping cpusets both have 'cpuset.sched_load_balance' enabled,
503 If, as is the default, the top cpuset has 'cpuset.sched_load_balance' enabled,
505 the whole system, regardless of any other cpuset settings.
510 of CPUs allowed to a cpuset having 'cpuset.sched_load_balance' enabled.
512 The internal kernel cpuset to scheduler interface passes from the
513 cpuset code to the scheduler code a partition of the load balanced
518 The cpuset code builds a new such partition and passes it to the
522 - the 'cpuset.sched_load_balance' flag of a cpuset with non-empty CPUs changes,
523 - or CPUs come or go from a cpuset with this flag enabled,
524 - or 'cpuset.sched_relax_domain_level' value of a cpuset with non-empty CPUs
526 - or a cpuset with non-empty CPUs and with this flag enabled is removed,
535 the cpuset code to update these sched domains, it compares the new
569 The 'cpuset.sched_relax_domain_level' file allows you to request changing
572 otherwise initial value -1 that indicates the cpuset has no request.
587 This file is per-cpuset and affect the sched domain where the cpuset
588 belongs to. Therefore if the flag 'cpuset.sched_load_balance' of a cpuset
589 is disabled, then 'cpuset.sched_relax_domain_level' have no effect since
590 there is no sched domain belonging the cpuset.
606 the searching cost enough small by managing cpuset to compact etc.
617 task directly, the impact on a task of changing its cpuset CPU
618 or Memory Node placement, or of changing to which cpuset a task
621 If a cpuset has its Memory Nodes modified, then for each task attached
622 to that cpuset, the next time that the kernel attempts to allocate
624 in the task's cpuset, and update its per-task memory placement to
627 its new cpuset, then the task will continue to use whatever subset
628 of MPOL_BIND nodes are still allowed in the new cpuset. If the task
630 in the new cpuset, then the task will be essentially treated as if it
631 was MPOL_BIND bound to the new cpuset (even though its NUMA placement,
633 from one cpuset to another, then the kernel will adjust the task's
637 If a cpuset has its 'cpuset.cpus' modified, then each task in that cpuset
639 if a task's pid is written to another cpuset's 'tasks' file, then its
641 bound to some subset of its cpuset using the sched_setaffinity() call,
642 the task will be allowed to run on any CPU allowed in its new cpuset,
645 In summary, the memory placement of a task whose cpuset is changed is
652 cpusets memory placement policy 'cpuset.mems' subsequently changes.
653 If the cpuset flag file 'cpuset.memory_migrate' is set true, then when
654 tasks are attached to that cpuset, any pages that task had
655 allocated to it on nodes in its previous cpuset are migrated
656 to the task's new cpuset. The relative placement of the page within
657 the cpuset is preserved during these migration operations if possible.
658 For example if the page was on the second valid node of the prior cpuset
659 then the page will be placed on the second valid node of the new cpuset.
661 Also if 'cpuset.memory_migrate' is set true, then if that cpuset's
662 'cpuset.mems' file is modified, pages allocated to tasks in that
663 cpuset, that were on nodes in the previous setting of 'cpuset.mems',
665 Pages that were not in the task's prior cpuset, or in the cpuset's
666 prior 'cpuset.mems' setting, will not be moved.
669 to remove all the CPUs that are currently assigned to a cpuset,
670 then all the tasks in that cpuset will be moved to the nearest ancestor
672 cpuset is bound with another cgroup subsystem which has some restrictions
674 in the original cpuset, and the kernel will automatically update
678 violate cpuset placement, over starving a task that has had all
685 the current task's cpuset, then we relax the cpuset, and look for
686 memory anywhere we can find it. It's better to violate the cpuset
689 To start a new job that is to be contained within a cpuset, the steps are:
691 1) mkdir /sys/fs/cgroup/cpuset
692 2) mount -t cgroup -ocpuset cpuset /sys/fs/cgroup/cpuset
693 3) Create the new cpuset by doing mkdir's and write's (or echo's) in
694 the /sys/fs/cgroup/cpuset virtual file system.
696 5) Attach that task to the new cpuset by writing its pid to the
697 /sys/fs/cgroup/cpuset tasks file for that cpuset.
700 For example, the following sequence of commands will setup a cpuset
702 and then start a subshell 'sh' in that cpuset::
704 mount -t cgroup -ocpuset cpuset /sys/fs/cgroup/cpuset
705 cd /sys/fs/cgroup/cpuset
708 /bin/echo 2-3 > cpuset.cpus
709 /bin/echo 1 > cpuset.mems
712 # The subshell 'sh' is now running in cpuset Charlie
714 cat /proc/self/cpuset
718 - via the cpuset file system directly, using the various cd, mkdir, echo,
724 (http://code.google.com/p/cpuset/)
737 Creating, modifying, using the cpusets can be done through the cpuset
741 # mount -t cgroup -o cpuset cpuset /sys/fs/cgroup/cpuset
743 Then under /sys/fs/cgroup/cpuset you can find a tree that corresponds to the
744 tree of the cpusets in the system. For instance, /sys/fs/cgroup/cpuset
745 is the cpuset that holds the whole system.
747 If you want to create a new cpuset under /sys/fs/cgroup/cpuset::
749 # cd /sys/fs/cgroup/cpuset
752 Now you want to do something with this cpuset::
759 cgroup.clone_children cpuset.memory_pressure
760 cgroup.event_control cpuset.memory_spread_page
761 cgroup.procs cpuset.memory_spread_slab
762 cpuset.cpu_exclusive cpuset.mems
763 cpuset.cpus cpuset.sched_load_balance
764 cpuset.mem_exclusive cpuset.sched_relax_domain_level
765 cpuset.mem_hardwall notify_on_release
766 cpuset.memory_migrate tasks
768 Reading them will give you information about the state of this cpuset:
771 the cpuset.
775 # /bin/echo 1 > cpuset.cpu_exclusive
779 # /bin/echo 0-7 > cpuset.cpus
783 # /bin/echo 0-7 > cpuset.mems
785 Now attach your shell to this cpuset::
789 You can also create cpusets inside your cpuset by using mkdir in this
794 To remove a cpuset, just use rmdir::
798 This will fail if the cpuset is in use (has cpusets inside, or has
801 Note that for legacy reasons, the "cpuset" filesystem exists as a
806 mount -t cpuset X /sys/fs/cgroup/cpuset
810 mount -t cgroup -ocpuset,noprefix X /sys/fs/cgroup/cpuset
811 echo "/sbin/cpuset_release_agent" > /sys/fs/cgroup/cpuset/release_agent
817 in cpuset directories::
819 # /bin/echo 1-4 > cpuset.cpus -> set cpus list to cpus 1,2,3,4
820 # /bin/echo 1,2,3,4 > cpuset.cpus -> set cpus list to cpus 1,2,3,4
822 To add a CPU to a cpuset, write the new list of CPUs including the
823 CPU to be added. To add 6 to the above cpuset::
825 # /bin/echo 1-4,6 > cpuset.cpus -> set cpus list to cpus 1,2,3,4,6
827 Similarly to remove a CPU from a cpuset, write the new list of CPUs
832 # /bin/echo "" > cpuset.cpus -> clear cpus list
839 # /bin/echo 1 > cpuset.cpu_exclusive -> set flag 'cpuset.cpu_exclusive'
840 # /bin/echo 0 > cpuset.cpu_exclusive -> unset flag 'cpuset.cpu_exclusive'
866 errors. If you use it in the cpuset file system, you won't be
879 Web: http://www.bullopensource.org/cpuset