Software performance is crucial for both CPU-based programs and modern large language model (LLM) applications. Several factors influence performance, with memory usage being a significant one. However, memory-related performance issues are often associated with various dependencies, including low-level software components and hardware elements. It is challenging to profile the impacts of those various dependencies precisely and efficiently. There are general-purpose profilers that utilize sampling methods to profile applications with low performance overhead. However, these tools mainly focus on the application itself and cannot identify issues arising from other dependencies. This dissertation systematically analyzes performance bottlenecks and introduces effective optimization techniques. We introduce CachePerf, a cache miss profiler to identify cache misses; and MemPerf, a profiler to detect issues from the memory allocator. These profilers successfully identify most issues related to cache misses and memory allocation, achieving performance speedups of up to 3788% while imposing minimal performance overhead. To analyze and optimize various types of software, we developed MemTrace, a memory analysis tool to profile dynamic memory management in autonomous driving (AD) software. We further propose Plasma, a framework for optimizing LLM inference with data transfer acceleration. Overall, this dissertation proposes methodologies that improve profiling efficiency and allow for scalable, adaptive performance optimizations in various computing environments.