kernel/tools/perf/util/session.h, branch linux-5.2.y

perf session: Fix loading of compressed data split across adjacent records

2019-08-16T08:11:05Z

[ Upstream commit 872c8ee8f0f47222f7b10da96eea84d0486540a3 ] Fix decompression failure found during the loading of compressed trace collected on larger scale systems (>48 cores). The error happened due to lack of decompression space for a mmaped buffer data chunk split across adjacent PERF_RECORD_COMPRESSED records. $ perf report -i bt.16384.data --stats failed to decompress (B): 63869 -> 0 : Destination buffer is too small user stack dump failure Can't parse sample, err = -14 0x2637e436 [0x4080]: failed to process type: 9 Error: failed to process sample $ perf test 71 71: Zstd perf.data compression/decompression : Ok Signed-off-by: Alexey Budankov Acked-by: Jiri Olsa Cc: Alexander Shishkin Cc: Andi Kleen Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/4d839e1b-9c48-89c4-9702-a12217420611@linux.intel.com Signed-off-by: Arnaldo Carvalho de Melo Signed-off-by: Sasha Levin

perf report: Implement perf.data record decompression

2019-05-15T19:36:49Z

zstd_init(, comp_level = 0) initializes decompression part of API only hat now consists of zstd_decompress_stream() function. The perf.data PERF_RECORD_COMPRESSED records are decompressed using zstd_decompress_stream() function into a linked list of mmaped memory regions of mmap_comp_len size (struct decomp). After decompression of one COMPRESSED record its content is iterated and fetched for usual processing. The mmaped memory regions with decompressed events are kept in the linked list till the tool process termination. When dumping raw records (e.g., perf report -D --header) file offsets of events from compressed records are printed as zero. Committer notes: Since now we have support for processing PERF_RECORD_COMPRESSED, we see none, in raw form, like we saw in the previous patch commiter notes, they were decompressed into the usual PERF_RECORD_{FORK,MMAP,COMM,etc} records, we only see the stats for those PERF_RECORD_COMPRESSED events, and since I used the file generated in the commiter notes for the previous patch, there they are, 2 compressed records: $ perf report --header-only | grep cmdline # cmdline : /home/acme/bin/perf record -z2 sleep 1 $ perf report -D | grep COMPRESS COMPRESSED events: 2 COMPRESSED events: 0 $ perf report --stdio # To display the perf.data header info, please use --header/--header-only options. # # # Total Lost Samples: 0 # # Samples: 15 of event 'cycles:u' # Event count (approx.): 962227 # # Overhead Command Shared Object Symbol # ........ ....... ................ ........................... # 46.99% sleep libc-2.28.so [.] _dl_addr 29.24% sleep [unknown] [k] 0xffffffffaea00a67 16.45% sleep libc-2.28.so [.] __GI__IO_un_link.part.1 5.92% sleep ld-2.28.so [.] _dl_setup_hash 1.40% sleep libc-2.28.so [.] __nanosleep 0.00% sleep [unknown] [k] 0xffffffffaea00163 # # (Tip: To see callchains in a more compact form: perf report -g folded) # $ Signed-off-by: Alexey Budankov Reviewed-by: Jiri Olsa Tested-by: Arnaldo Carvalho de Melo Cc: Alexander Shishkin Cc: Andi Kleen Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/304b0a59-942c-3fe1-da02-aa749f87108b@linux.intel.com Signed-off-by: Arnaldo Carvalho de Melo

perf record: Implement compression for serial trace streaming

2019-05-15T19:36:49Z

Compression is implemented using the functions from zstd.c. As the memory to operate on the compression uses mmap->data buffer. If Zstd streaming compression API fails for some reason the data to be compressed are just copied into the memory buffers using plain memcpy(). Compressed trace frame consists of an array of PERF_RECORD_COMPRESSED records. Each element of the array is not longer that PERF_SAMPLE_MAX_SIZE and consists of perf_event_header followed by the compressed chunk that is decompressed on the loading stage. Comitter notes: Undo some unnecessary line breaks, remove some unnecessary () around zstd_data to then just get its address, and fix conflicts with BPF_PROG_INFO/BPF_BTF patchkits. Signed-off-by: Alexey Budankov Reviewed-by: Jiri Olsa Cc: Alexander Shishkin Cc: Andi Kleen Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/744df43f-3932-2594-ddef-1e99a3cad03a@linux.intel.com Signed-off-by: Arnaldo Carvalho de Melo

perf session: Define 'bytes_transferred' and 'bytes_compressed' metrics

2019-05-15T19:36:49Z

Define 'bytes_transferred' and 'bytes_compressed' metrics to calculate ratio in the end of the data collection: compression ratio = bytes_transferred / bytes_compressed The 'bytes_transferred' metric accumulates the amount of bytes that was extracted from the mmaped kernel buffers for compression, while 'bytes_compressed' accumulates the amount of bytes that was received after applying compression. Signed-off-by: Alexey Budankov Reviewed-by: Jiri Olsa Cc: Alexander Shishkin Cc: Andi Kleen Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/1d4bf499-cb03-26dc-6fc6-f14fec7622ce@linux.intel.com Signed-off-by: Arnaldo Carvalho de Melo

perf tools: Remove perf_tool from event_op2

2018-09-19T13:25:10Z

Now that we keep a perf_tool pointer inside perf_session, there's no need to have a perf_tool argument in the event_op2 callback. Remove it. Signed-off-by: Jiri Olsa Cc: Alexander Shishkin Cc: Alexey Budankov Cc: Andi Kleen Cc: Namhyung Kim Cc: Peter Zijlstra Link: http://lkml.kernel.org/r/20180913125450.21342-2-jolsa@kernel.org Signed-off-by: Arnaldo Carvalho de Melo

Merge tag 'perf-core-for-mingo-4.15-20171117' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core

2017-11-18T07:59:27Z

Pull perf/core improvements and fixes from Arnaldo Carvalho de Melo: - Optimize sample parsing for ordering events, where we don't need to parse all the PERF_SAMPLE_ bits, just the ones leading to the timestamp needed to reorder events (Jiri Olsa) - Use a dummy event to ask for PERF_RECORD_{MMAP,COMM,EXEC} with 'perf record --delay', when the events asked by the user will only be enabled after the workload is started and the requested delay passes, so we need to add the dummy event and have it .enabled_on_exec. This then allows us to resolve symbols for the DSO executable MMAPs setup while we wait for the delay (Arnaldo Carvalho de Melo) - Synchronize kcmp.h and prctl.h ABI headers wrt SPDX tags (Arnaldo Carvalho de Melo) - Generalize the annotation code to support other source information besides objdump/DWARF obtained ones, starting with python scripts, that will is slated to be merged soon (Jiri Olsa) - Advance the source code lines to right after the column with the address in asm lines (Jiri Olsa) - Fix terminal dimensions resizing signal handling in 'perf top --stdio' (Jiri Olsa) - Improve error messages for PMU events (Kim Phillips) - Fix 'perf record' -c/-F options for cpu event aliases (Andi Kleen) - Enable type checking for perf_evsel_config_term types (Andi Kleen) - Call machine__exit() at 'perf trace' exit, so as to remove temporary files related to VDSO (Andrei Vagin) - Add "reject" option to parse-events.l, fixing the build with newer flex releases. Noticed with flex 2.6.4 on Alpine Linux 3.6 and Edge (Jiri Olsa) - Document some missing perf.data headers (Andi Kleen) - Allow printing period for non freq mod groups (Andi Kleen) - Do not warn the user about kernel.kptr_restrict when not sampling the kernel (Arnaldo Carvalho de Melo) - Fix bug in 'perf help' introduced during conversion to strstart() (Namhyung Kim) - Do not truncate ASM instruction mnemonics at 6 characters in the annotation output, PowerPC has long ones (Ravi Bangoria) - Document some missing command line options (Sihyeon Jang) - Update POWER9 vendor event tables (Sukadev Bhattiprolu) - Fix 'perf test' shell entries on s390x, where the 'openat' syscall is used instead of 'open' in one of the tests and - No need to use overwrite mmap mode in 'perf test', those tests do not generate massive amount of events to fill the ring buffer (Wang Nan) - Add missing command line options (mostly --force/-f) to the man pages (Sihyeon Jang) Signed-off-by: Arnaldo Carvalho de Melo Signed-off-by: Ingo Molnar

perf ordered_events: Pass timestamp arg in perf_session__queue_event

2017-11-17T15:14:09Z

There's no need to pass whole sample data, because it's only timestamp that is used. Signed-off-by: Jiri Olsa Cc: Adrian Hunter Cc: David Ahern Cc: Jiri Olsa Cc: Namhyung Kim Cc: Wang Nan Link: http://lkml.kernel.org/n/tip-xd1hpoze3kgb1rb639o3vehb@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo

Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip

2017-11-13T21:05:08Z

Pull perf updates from Ingo Molnar: "The main changes in this cycle were: Kernel: - kprobes updates: use better W^X patterns for code modifications, improve optprobes, remove jprobes. (Masami Hiramatsu, Kees Cook) - core fixes: event timekeeping (enabled/running times statistics) fixes, perf_event_read() locking fixes and cleanups, etc. (Peter Zijlstra) - Extend x86 Intel free-running PEBS support and support x86 user-register sampling in perf record and perf script. (Andi Kleen) Tooling: - Completely rework the way inline frames are handled. Instead of querying for the inline nodes on-demand in the individual tools, we now create proper callchain nodes for inlined frames. (Milian Wolff) - 'perf trace' updates (Arnaldo Carvalho de Melo) - Implement a way to print formatted output to per-event files in 'perf script' to facilitate generate flamegraphs, elliminating the need to write scripts to do that separation (yuzhoujian, Arnaldo Carvalho de Melo) - Update vendor events JSON metrics for Intel's Broadwell, Broadwell Server, Haswell, Haswell Server, IvyBridge, IvyTown, JakeTown, Sandy Bridge, Skylake, SkyLake Server - and Goldmont Plus V1 (Andi Kleen, Kan Liang) - Multithread the synthesizing of PERF_RECORD_ events for pre-existing threads in 'perf top', speeding up that phase, greatly improving the user experience in systems such as Intel's Knights Mill (Kan Liang) - Introduce the concept of weak groups in 'perf stat': try to set up a group, but if it's not schedulable fallback to not using a group. That gives us the best of both worlds: groups if they work, but still a usable fallback if they don't. E.g: (Andi Kleen) - perf sched timehist enhancements (David Ahern) - ... various other enhancements, updates, cleanups and fixes" * 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (139 commits) kprobes: Don't spam the build log with deprecation warnings arm/kprobes: Remove jprobe test case arm/kprobes: Fix kretprobe test to check correct counter perf srcline: Show correct function name for srcline of callchains perf srcline: Fix memory leak in addr2inlines() perf trace beauty kcmp: Beautify arguments perf trace beauty: Implement pid_fd beautifier tools include uapi: Grab a copy of linux/kcmp.h perf callchain: Fix double mapping al->addr for children without self period perf stat: Make --per-thread update shadow stats to show metrics perf stat: Move the shadow stats scale computation in perf_stat__update_shadow_stats perf tools: Add perf_data_file__write function perf tools: Add struct perf_data_file perf tools: Rename struct perf_data_file to perf_data perf script: Print information about per-event-dump files perf trace beauty prctl: Generate 'option' string table from kernel headers tools include uapi: Grab a copy of linux/prctl.h perf script: Allow creating per-event dump files perf evsel: Restore evsel->priv as a tool private area perf script: Use event_format__fprintf() ...

Merge branch 'linus' into locking/core, to resolve conflicts

2017-11-07T09:32:44Z

Conflicts: include/linux/compiler-clang.h include/linux/compiler-gcc.h include/linux/compiler-intel.h include/uapi/linux/stddef.h Signed-off-by: Ingo Molnar

Merge branch 'linus' into perf/core, to fix conflicts

2017-11-07T09:30:18Z

Conflicts: tools/perf/arch/arm/annotate/instructions.c tools/perf/arch/arm64/annotate/instructions.c tools/perf/arch/powerpc/annotate/instructions.c tools/perf/arch/s390/annotate/instructions.c tools/perf/arch/x86/tests/intel-cqm.c tools/perf/ui/tui/progress.c tools/perf/util/zlib.c Signed-off-by: Ingo Molnar