kernel/tools/perf/util/event.c, branch linux-4.1.y

perf tools: Make perf_event__synthesize_mmap_events() scale

2018-05-23T01:33:43Z

[ Upstream commit 88b897a30c525c2eee6e7f16e1e8d0f18830845e ] This patch significantly improves the execution time of perf_event__synthesize_mmap_events() when running perf record on systems where processes have lots of threads. It just happens that cat /proc/pid/maps support uses a O(N^2) algorithm to generate each map line in the maps file. If you have 1000 threads, then you have necessarily 1000 stacks. For each vma, you need to check if it corresponds to a thread's stack. With a large number of threads, this can take a very long time. I have seen latencies >> 10mn. As of today, perf does not use the fact that a mapping is a stack, therefore we can work around the issue by using /proc/pid/tasks/pid/maps. This entry does not try to map a vma to stack and is thus much faster with no loss of functonality. The proc-map-timeout logic is kept in case users still want some upper limit. In V2, we fix the file path from /proc/pid/tasks/pid/maps to actual /proc/pid/task/pid/maps, tasks -> task. Thanks Arnaldo for catching this. Committer note: This problem seems to have been elliminated in the kernel since commit : b18cb64ead40 ("fs/proc: Stop trying to report thread stacks"). Signed-off-by: Stephane Eranian Acked-by: Jiri Olsa Cc: Andy Lutomirski Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20170315135059.GC2177@redhat.com Link: http://lkml.kernel.org/r/1489598233-25586-1-git-send-email-eranian@google.com Signed-off-by: Arnaldo Carvalho de Melo Signed-off-by: Sasha Levin

perf tools: Use readdir() instead of deprecated readdir_r()

2017-07-31T17:37:53Z

[ Upstream commit 7093b4c963cc4e344e490c774924a180602a7092 ] The readdir() function is thread safe as long as just one thread uses a DIR, which is the case when synthesizing events for pre-existing threads by traversing /proc, so, to avoid breaking the build with glibc-2.23.90 (upcoming 2.24), use it instead of readdir_r(). See: http://man7.org/linux/man-pages/man3/readdir.3.html "However, in modern implementations (including the glibc implementation), concurrent calls to readdir() that specify different directory streams are thread-safe. In cases where multiple threads must read from the same directory stream, using readdir() with external synchronization is still preferable to the use of the deprecated readdir_r(3) function." Noticed while building on a Fedora Rawhide docker container. CC /tmp/build/perf/util/event.o util/event.c: In function '__event__synthesize_thread': util/event.c:466:2: error: 'readdir_r' is deprecated [-Werror=deprecated-declarations] while (!readdir_r(tasks, &dirent, &next) && next) { ^~~~~ In file included from /usr/include/features.h:368:0, from /usr/include/stdint.h:25, from /usr/lib/gcc/x86_64-redhat-linux/6.0.0/include/stdint.h:9, from /git/linux/tools/include/linux/types.h:6, from util/event.c:1: /usr/include/dirent.h:189:12: note: declared here Cc: Adrian Hunter Cc: David Ahern Cc: Jiri Olsa Cc: Namhyung Kim Cc: Wang Nan Link: http://lkml.kernel.org/n/tip-i1vj7nyjp2p750rirxgrfd3c@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo Signed-off-by: Sasha Levin

perf tools: handle spaces in file names obtained from /proc/pid/maps

2016-07-11T03:07:15Z

[ Upstream commit 89fee59b504f86925894fcc9ba79d5c933842f93 ] Steam frequently puts game binaries in folders with spaces. Note: "(deleted)" markers are now treated as part of the file name. Signed-off-by: Marcin Ślusarz Acked-by: Namhyung Kim Fixes: 6064803313ba ("perf tools: Use sscanf for parsing /proc/pid/maps") Link: http://lkml.kernel.org/r/20160119190303.GA17579@marcin-Inspiron-7720 Signed-off-by: Arnaldo Carvalho de Melo Signed-off-by: Sasha Levin

perf tools: Fix error path to do closedir() when synthesizing threads

2015-04-10T13:13:58Z

When traversing /proc to synthesize the PERF_RECORD_FORK et al events we were bailing out on errors without calling closedir(), fix it. Reported-by: David Ahern Cc: Adrian Hunter Cc: Borislav Petkov Cc: Don Zickus Cc: Frederic Weisbecker Cc: Jiri Olsa Cc: Mike Galbraith Cc: Namhyung Kim Cc: Peter Zijlstra Cc: Stephane Eranian Link: http://lkml.kernel.org/n/tip-vxtp593rfztgbi8noy0m967p@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Fix synthesizing fork_event.ppid for non-main thread

2015-04-10T13:10:55Z

Commit ca6c41c59b9 sets the ppid based on what is read from the /proc/pid/status file when synthesizing fork events. This is correct thing to do for new processes but not threads of a process. Fix ppid for threads to be the main thread when synthesizing fork events (ie., assume main thread spawned all sub-threads in a process). Reported-by: Arnaldo Carvalho de Melo Signed-off-by: David Ahern Tested-by: Arnaldo Carvalho de Melo Acked-by: Don Zickus Link: http://lkml.kernel.org/r/1428598107-178999-1-git-send-email-david.ahern@oracle.com Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Fix ppid for synthesized fork events

2015-03-31T20:52:30Z

363b785f38 added synthesized fork events and set a thread's parent id to itself. Since we are already processing /proc//status the ppid can be determined properly. Make it so. Signed-off-by: David Ahern Acked-by: Don Zickus Acked-by: Jiri Olsa Cc: Joe Mario Link: http://lkml.kernel.org/r/1427747758-18510-2-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Refactor comm/tgid lookup

2015-03-31T20:52:30Z

Rather than parsing /proc/pid/status file one line at a time, read it into a buffer in one shot and search for all strings in one pass. tgid conversion also simplified -- removing the isspace walk. As noted by Arnaldo those are not needed for atoi == strtol calls. Signed-off-by: David Ahern Acked-by: Don Zickus Acked-by: Jiri Olsa Cc: Joe Mario Link: http://lkml.kernel.org/r/1427747758-18510-1-git-send-email-dsahern@gmail.com Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Fix FORK after COMM when synthesizing records for pre-existing threads

2015-03-02T14:51:30Z

In this commit: commit 363b785f3805a2632eb09a8b430842461c21a640 Author: Don Zickus Date: Fri Mar 14 10:43:44 2014 -0400 perf tools: Speed up thread map generation We ended up emitting PERF_RECORD_FORK events after their corresponding PERF_RECORD_COMM, so the code below will remove the "existing thread" and then recreates it, unnecessarily: [root@ssdandy ~]# perf probe -x ~/bin/perf -L machine__process_fork_event 0 int machine__process_fork_event(struct machine *machine, union perf_event *event, struct perf_sample *sample) 2 { 3 struct thread *thread = machine__find_thread(machine, event->fork.pid, event->fork.tid); 6 struct thread *parent = machine__findnew_thread(machine, event->fork.ppid, event->fork.ptid); /* if a thread currently exists for the thread id remove it */ if (thread != NULL) 12 machine__remove_thread(machine, thread); 14 thread = machine__findnew_thread(machine, event->fork.pid, event->fork.tid); 16 if (dump_trace) 17 perf_event__fprintf_task(event, stdout); 19 if (thread == NULL || parent == NULL || 20 thread__fork(thread, parent, sample->time) < 0) { 21 dump_printf("problem processing PERF_RECORD_FORK, skipping event.\n"); 22 return -1; } 25 return 0; 26 } [root@ssdandy ~]# perf probe -x ~/bin/perf fork_after_comm=machine__process_fork_event:12 Added new event: probe_perf:fork_after_comm (on machine__process_fork_event:12 in /home/acme/bin/perf) You can now use it in all perf tools, such as: perf record -e probe_perf:fork_after_comm -aR sleep 1 [root@ssdandy ~]# [root@ssdandy ~]# perf record -g -e probe_perf:* trace -o /tmp/bla ^C[ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.021 MB perf.data (30 samples) ] Terminated [root@ssdandy ~]# [root@ssdandy ~]# perf report --no-children --show-total-period --stdio # To display the perf.data header info, please use --header/--header-only options. # # Samples: 30 of event 'probe_perf:fork_after_comm' # Event count (approx.): 30 # # Overhead Period Command Shared Object Symbol # ........ ............ ....... ............. ............................... # 100.00% 30 trace trace [.] machine__process_fork_event | ---machine__process_fork_event __event__synthesize_thread.part.2 perf_event__synthesize_threads cmd_trace main __libc_start_main [root@ssdandy ~]# And Looking at 'perf report -D' output we see it: 0 0 0x8698 [0x30]: PERF_RECORD_COMM: auditd:703/707 0 0 0x86c8 [0x38]: PERF_RECORD_FORK(703:707):(703:703) Fix it by more closely mimicking how the kernel generates those records when a new fork happens, i.e. first a PERF_RECORD_FORK, then a PERF_RECORD_COMM. Cc: Adrian Hunter Cc: Borislav Petkov Cc: David Ahern Cc: Don Zickus Cc: Frederic Weisbecker Cc: Jiri Olsa Cc: Namhyung Kim Cc: Stephane Eranian Link: http://lkml.kernel.org/n/tip-h0emvymi2t3mw8dlqd6d6z73@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Print the thread's tid on PERF_RECORD_COMM events when -D is asked

2015-02-24T20:34:00Z

Cc: Adrian Hunter Cc: Borislav Petkov Cc: David Ahern Cc: Don Zickus Cc: Frederic Weisbecker Cc: Jiri Olsa Cc: Namhyung Kim Cc: Stephane Eranian Link: http://lkml.kernel.org/n/tip-fmto8ft6jrtwz09dxn5d4z8w@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Add id index

2014-10-29T13:24:47Z

Add an index of the event identifiers, in preparation for Intel PT. The event id (also called the sample id) is a unique number allocated by the kernel to the event created by perf_event_open(). Events can include the event id by having a sample type including PERF_SAMPLE_ID or PERF_SAMPLE_IDENTIFIER. Currently the main use of the event id is to match an event back to the evsel to which it belongs i.e. perf_evlist__id2evsel() The purpose of this patch is to make it possible to match an event back to the mmap from which it was read. The reason that is useful is because the mmap represents a time-ordered context (either for a cpu or for a thread). Intel PT decodes trace information on that basis. In full-trace mode, that information can be recorded when the Intel PT trace is read, but in sample-mode the Intel PT trace data is embedded in a sample and it is in that case that the "id index" is needed. So the mmaps are numbered (idx) and the cpu and tid recorded against the id by perf_evlist__set_sid_idx() which is called by perf_evlist__mmap_per_evsel(). That information is recorded on the perf.data file in the new "id index". idx, cpu and tid are added to struct perf_sample_id (which is the node of evlist's hash table to match ids to evsels). The information can be retrieved using perf_evlist__id2sid(). Note however this all depends on having a sample type including PERF_SAMPLE_ID or PERF_SAMPLE_IDENTIFIER, otherwise ids are not recorded. The "id index" is a synthesized event record which will be created when Intel PT sampling is used by calling perf_event__synthesize_id_index(). Signed-off-by: Adrian Hunter Acked-by: Jiri Olsa Cc: David Ahern Cc: Frederic Weisbecker Cc: Jiri Olsa Cc: Namhyung Kim Cc: Paul Mackerras Cc: Peter Zijlstra Cc: Stephane Eranian Link: http://lkml.kernel.org/r/1414417770-18602-2-git-send-email-adrian.hunter@intel.com Signed-off-by: Arnaldo Carvalho de Melo