Pytorch Profiler

document
visual profiler w/ Chrome trace. (JSON파일을 여기에 drop하면 볼 수 있음.)
memory copies, kernel launches, flow event에 대한 정보 제공. 커널 성능 혹은 개선 방법에 대한 정보는 제공하지 않음

Profiling `torch.square()`

send tensor to GPU
Compute square of the tensor

Default usage

gpu-practice/profile/pt_profiler-default.py

Lines 5 to 10 in 76804ca

    
           # ## Default way to use profiler 
        
           with profile(activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA]) as prof: 
        
               for _ in range(10): 
        
                   a = torch.square(torch.randn(10000, 10000).cuda()) 
        
           prof.export_chrome_trace("trace.json")

Profiling Results

custom usage - w/ schedule

gpu-practice/profile/pt_profiler-custom.py

Lines 5 to 42 in 76804ca

    
           ## With warmup and skip 
        
           # https://pytorch.org/docs/stable/profiler.html 
        
           # Non-default profiler schedule allows user to turn profiler on and off 
        
           # on different iterations of the training loop; 
        
           # trace_handler is called every time a new trace becomes available 
        
           def trace_handler(prof): 
        
               print(prof.key_averages().table( 
        
                   sort_by="self_cuda_time_total", row_limit=-1)) 
        
               prof.export_chrome_trace("/tmp/test_trace_" + str(prof.step_num) + ".json") 
        
           with torch.profiler.profile( 
        
               activities=[ 
        
                   torch.profiler.ProfilerActivity.CPU, 
        
                   torch.profiler.ProfilerActivity.CUDA, 
        
               ], 
        
               # In this example with wait=1, warmup=1, active=2, repeat=1, 
        
               # profiler will skip the first step/iteration, 
        
               # start warming up on the second, record 
        
               # the third and the forth iterations, 
        
               # after which the trace will become available 
        
               # and on_trace_ready (when set) is called; 
        
               # the cycle repeats starting with the next step 
        
               schedule=torch.profiler.schedule( 
        
                   wait=1, 
        
                   warmup=1, 
        
                   active=2, 
        
                   repeat=1), 
        
               on_trace_ready=trace_handler 
        
               # on_trace_ready=torch.profiler.tensorboard_trace_handler('./log') 
        
               # used when outputting for tensorboard 
        
               ) as p: 
        
                   for iter in range(10): 
        
                       torch.square(torch.randn(10000, 10000).cuda()) 
        
                       # send a signal to the profiler that the next iteration has started 
        
                       p.step()

Profiling Results

나는 왜 HtoD copy가 안뜨지 ㅠㅠ.. profilerStop*이전 단계가 내꺼에선 안뜨는 문제 ㅠ

	# ## Default way to use profiler
	with profile(activities=[ProfilerActivity.CPU, ProfilerActivity.CUDA]) as prof:
	for _ in range(10):
	a = torch.square(torch.randn(10000, 10000).cuda())

	prof.export_chrome_trace("trace.json")

	## With warmup and skip
	# https://pytorch.org/docs/stable/profiler.html

	# Non-default profiler schedule allows user to turn profiler on and off
	# on different iterations of the training loop;
	# trace_handler is called every time a new trace becomes available
	def trace_handler(prof):
	print(prof.key_averages().table(
	sort_by="self_cuda_time_total", row_limit=-1))
	prof.export_chrome_trace("/tmp/test_trace_" + str(prof.step_num) + ".json")

	with torch.profiler.profile(
	activities=[
	torch.profiler.ProfilerActivity.CPU,
	torch.profiler.ProfilerActivity.CUDA,
	],

	# In this example with wait=1, warmup=1, active=2, repeat=1,
	# profiler will skip the first step/iteration,
	# start warming up on the second, record
	# the third and the forth iterations,
	# after which the trace will become available
	# and on_trace_ready (when set) is called;
	# the cycle repeats starting with the next step

	schedule=torch.profiler.schedule(
	wait=1,
	warmup=1,
	active=2,
	repeat=1),
	on_trace_ready=trace_handler
	# on_trace_ready=torch.profiler.tensorboard_trace_handler('./log')
	# used when outputting for tensorboard
	) as p:
	for iter in range(10):
	torch.square(torch.randn(10000, 10000).cuda())
	# send a signal to the profiler that the next iteration has started
	p.step()

Pytorch Profiler

Profiling torch.square()

Default usage

Profiling Results

custom usage - w/ schedule

Profiling Results

Profiling `torch.square()`

How to profile CUDA kernels in PyTorch #1

Description

Contents

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions