If I understand your requirement correctly, your select clause is flawed. Instead of using 2 subqueries in your where clause, use a join between the two tables in the subquery. Also, you can't embed the object itself in your SQL string, you need to concatenate:
SELECT *
FROM youtube
WHERE id NOT IN(SELECT site_id
FROM youtube INNER JOIN viewed
ON youtube.id=viewed.site_id
AND youtube.user=viewed.user_id
WHERE user_id=" & ListView1.Items(0).SubItems(0).Text & ")
I'd strongly advise switching to a command object and parameterizing your query to avoid SQL injection.