In the book, implementations of basic algorithms of linear algebra are studied for most popular architectures of today's computers and tools for their implementations are discussed. The first chapter contains a discussion of memory and data dependencies. The second chapter presents some applications (e.g., LAPACK, sparse iterative methods, FFT and Monte Carlo methods). Their parallel implementations are discussed. The third chapter is devoted to SIMD processors, and the fourth chapter studies shared memory computers. OpenMP is also described. The fifth chapter explains MIMD architectures; MPI and PETSc tools are presented. The book has seven appendices. Appendix A describes SSE instructions of Intel processors, and Appendix B describes the Altivec intrinsic for the Motorola/IBM/Apple G-4 chip. Appendix C shows OpenMP commands and Appendix D presents MPI commands. Communication between C and FORTRAN codes is shown in Appendix E. The book will be of interest to students and researchers in computer science, applied and numerical mathematics, who deal with design and implementation of parallel codes.