In general, applications in macOS are packaged into a ".app" bundle that appears to be a single file but is actually a self-contained folder. Unlike Windows, in which an application usually installs a folder that contains the executable and supporting files, most of what an macOS app needs to run is stored within the .app bundle. Deleting an application bundle will remove that application's binary and all the supporting files contained within. Many apps, however, also install additional files in the user's Library folder, such as application preferences and caches.
To manually remove an macOS app, make sure the app is closed and head to the user's Library folder (in OS X Lion and above, hold down the Option key while selecting the Go menu from the Finder's menu bar and select Library). Here, you'll want to check for references to the application in the Application Support, Caches, LaunchAgents, and Preferences folders. Remove any files or folders that you are certain belong to the application you're trying to uninstall. You may also want to check the main Library folder by navigating to the top level of your hard drive and opening the /Library folder, although most applications will confine their files to the user-specific Library. Due to sandboxing requirements imposed by Apple, apps obtained from the Mac App Store are even easier to remove. Simply delete two items: the application file itself from the Applications folder and the application-specific folder in ~/Library/Containers (user's Library folder).
Another way to remove macOS apps is to use third-party tools. Take note, however, that automated tools can sometimes miss certain files or folders, and users employing these tools should always perform a manual check to ensure that all remnants of the application have indeed been removed. Finally, some applications (such as Cocktail) have their own uninstaller. Wherever possible, use the application-specific uninstaller for these applications.
Regardless of which method you choose, remember that leaving behind the occasional abandoned preference file is not likely to cause harm or performance issues. In general, removing the app bundle from the Applications folder and a file or two from the user's Library folder is enough to remove the application from the drive and free up disk space.